03:02:19,856 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:19,872 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2023494382064820608] 03:02:19,873 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:19,880 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:19,881 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:19,887 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:19,887 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:19,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:19,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:19,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:19,911 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:19,931 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [790739844027828250] 03:02:19,933 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:19,936 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:19,936 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:19,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:19,942 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:19,966 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:19,982 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2103615687436600780] 03:02:19,982 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:19,990 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:19,990 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:19,992 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:19,992 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:20,0 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:20,20 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:20,26 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 03:02:20,60 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 03:02:20,61 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1683608257] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,82 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 03:02:20,82 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,83 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 03:02:20,83 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:02:20,83 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,83 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@28d3e196 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6dd17ae8 03:02:20,84 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@13d3a10e 03:02:20,84 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1709823740083, None)) 03:02:20,85 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823740082,0,0)) 03:02:20,96 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:20,96 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,96 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,96 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@30367571 03:02:20,101 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:20,105 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:20,114 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:20,119 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6dd17ae8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4d0fb453 03:02:20,120 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:20,120 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 03:02:20,126 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:20,126 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:20,126 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,130 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:20,131 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,131 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,131 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268] 03:02:20,131 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,131 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,131 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,131 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550] 03:02:20,132 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 03:02:20,132 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 03:02:20,133 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 03:02:20,133 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 03:02:20,134 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 03:02:20,138 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,138 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,138 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#-563454978 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,138 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#-563454978] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,138 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#887237100] 03:02:20,138 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#1461237842 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,138 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1339620437] 03:02:20,138 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#887237100] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,142 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 03:02:20,143 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:20,143 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:20,148 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 03:02:20,148 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 03:02:20,148 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:20,148 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#1461237842] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,148 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:20,148 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1339620437] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,152 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 03:02:20,152 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:20,152 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 03:02:20,152 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 03:02:20,152 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 03:02:20,152 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 03:02:20,153 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 03:02:20,161 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-206976006] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,168 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:20,168 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 03:02:20,169 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 03:02:20,169 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 03:02:20,169 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 03:02:20,170 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 03:02:20,170 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:02:20,170 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@200dbe48 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@75104d0b 03:02:20,170 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1167fcf6 03:02:20,171 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 03:02:20,171 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,171 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,172 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1709823740171, None)) 03:02:20,172 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823740170,0,0)) 03:02:20,172 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:20,172 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,172 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,172 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@75104d0b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@170918e9 03:02:20,172 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 03:02:20,173 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4641c9cf 03:02:20,174 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:20,174 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,174 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,174 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,174 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-2-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#850163478] 03:02:20,174 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,175 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 03:02:20,175 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#850163478] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,175 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#743173747] 03:02:20,176 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1045254795 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,176 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#1045254795] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,176 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#743173747] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,176 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 03:02:20,176 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 03:02:20,176 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#173451109] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,177 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#-1991593138 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,177 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-753625551] 03:02:20,177 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#-1991593138] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,177 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-753625551] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:20,177 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 03:02:20,178 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,178 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,178 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-2-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#173451109] 03:02:20,178 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:20,179 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:20,179 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:20,179 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 03:02:20,179 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,179 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 03:02:20,179 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 03:02:20,179 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 03:02:20,179 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 03:02:20,180 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 03:02:20,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:20,180 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:20,180 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 03:02:20,180 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 03:02:20,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:20,180 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 03:02:20,181 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 03:02:20,181 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,181 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 03:02:20,181 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 03:02:20,223 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1683608257] 03:02:20,223 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 03:02:20,223 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268] 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 03:02:20,432 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 03:02:20,432 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 03:02:20,452 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 03:02:20,452 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 03:02:20,681 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:02:20,681 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,681 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 03:02:20,711 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:20,742 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:02:20,742 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,742 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 03:02:20,791 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:20,904 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:20,904 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,905 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,923 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:02:20,939 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:02:20,939 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,939 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 03:02:20,939 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:20,950 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:20,950 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 03:02:20,950 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:20,950 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:20,950 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=null 03:02:20,950 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:20,950 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,950 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:02:20,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:02:20,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:02:20,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 03:02:20,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:20,958 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:02:20,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:02:20,958 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 03:02:20,958 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 03:02:20,958 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 03:02:20,959 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:20,959 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:20,959 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 03:02:20,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 03:02:20,973 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:20,974 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 03:02:20,974 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:20,974 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:20,974 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 03:02:20,984 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:02:20,984 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:21,2 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:21,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:02:21,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:02:21,31 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 03:02:21,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 03:02:21,33 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1683608257] 03:02:21,33 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 03:02:21,33 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550] 03:02:21,36 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:21,37 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 03:02:21,37 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:21,37 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:21,37 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=null 03:02:21,37 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:21,37 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:02:21,37 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:02:21,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:02:21,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:02:21,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 03:02:21,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:21,42 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:02:21,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:02:21,42 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 03:02:21,43 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 03:02:21,47 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 03:02:21,47 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:02:21,48 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 03:02:21,48 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 03:02:21,48 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:21,48 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:21,48 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 03:02:21,53 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:02:21,53 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:21,53 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 03:02:21,53 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:21,53 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:21,53 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 03:02:21,53 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 03:02:21,70 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,71 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-206976006] 03:02:21,71 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:21,72 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 03:02:21,72 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:21,72 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-206976006] 03:02:21,72 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 03:02:21,72 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:21,72 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:02:21,72 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 03:02:21,72 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 03:02:21,72 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:21,72 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:21,72 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 03:02:21,72 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars 03:02:21,72 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 03:02:21,72 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 03:02:21,72 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:21,73 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 03:02:21,73 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:21,75 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 03:02:21,75 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:02:21,75 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 03:02:21,75 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 03:02:21,75 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 03:02:21,75 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 03:02:21,75 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 03:02:21,76 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 03:02:21,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 03:02:21,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:21,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:21,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 03:02:21,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 03:02:21,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 03:02:21,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,81 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,82 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:02:21,82 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:21,82 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@340d7ec2 03:02:21,82 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,82 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 03:02:21,82 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,82 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 03:02:21,82 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 03:02:21,83 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 03:02:21,83 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,83 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@78302520 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 03:02:21,84 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:02:21,85 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,86 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,91 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 1 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 1 03:02:21,92 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:21,93 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 03:02:21,93 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] resolved 03:02:21,93 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:02:21,93 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 03:02:21,93 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] resolved 03:02:21,93 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: successfully resolved all cohort actors 03:02:21,93 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finishCanCommit 03:02:21,95 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] 03:02:21,95 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] 03:02:21,96 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:02:21,96 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,96 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 03:02:21,96 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 03:02:21,96 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=17186364} is already disabled 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 03:02:21,98 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=15313000} is already disabled 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:21,99 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 03:02:21,106 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:21,106 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] 03:02:21,106 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:02:21,107 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:02:21,108 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,108 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:02:21,108 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 03:02:21,108 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:21,116 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:21,117 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 03:02:21,121 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:02:21,121 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:21,121 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit returning result: true 03:02:21,122 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 commit 03:02:21,122 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finish commit 03:02:21,122 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] 03:02:21,122 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] 03:02:21,122 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] 03:02:21,122 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 03:02:21,127 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}]]] 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 03:02:21,128 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:21,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:21,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:21,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}]]] 03:02:21,129 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 03:02:21,130 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 03:02:21,131 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}]]] 03:02:21,131 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}]]] 03:02:21,131 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:21,131 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:21,131 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:02:21,131 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:02:21,131 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=180} 03:02:21,131 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=176} 03:02:21,131 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 03:02:21,131 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 03:02:21,131 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 03:02:21,131 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 03:02:21,131 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:21,131 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:21,132 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 03:02:21,132 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 03:02:21,136 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%23-1090524550$f] 03:02:21,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1194719392] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 03:02:21,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 03:02:21,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:21,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 03:02:21,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 03:02:21,137 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:21,137 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1194719392] 03:02:21,137 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,138 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1194719392] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:21,138 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 106.7 μs 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=176}] 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%23285506268$e] 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1316241089] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 03:02:21,138 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1316241089] 03:02:21,138 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 03:02:21,138 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,139 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 03:02:21,139 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 71.91 μs 03:02:21,139 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1316241089] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:21,141 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 03:02:21,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:02:21,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 03:02:21,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 03:02:21,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 03:02:21,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1138699244] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 03:02:21,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 03:02:21,142 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 03:02:21,143 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit succeeded 03:02:21,143 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1138699244] 03:02:21,143 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,144 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 103.5 μs 03:02:21,144 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,144 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1138699244] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:21,144 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:21,144 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:21,144 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 03:02:21,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 03:02:21,145 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 03:02:21,146 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 03:02:21,146 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:21,146 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 03:02:21,146 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 03:02:21,146 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 03:02:21,146 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 03:02:21,155 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,155 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:21,155 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5f1f8621 03:02:21,155 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,155 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:21,155 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7fb3e3c6 03:02:21,156 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-24336561] 03:02:21,156 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1966670640] 03:02:21,156 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-24336561] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:21,162 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1966670640, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] 03:02:21,162 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-24336561, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] 03:02:21,162 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 03:02:21,162 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:02:21,162 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:02:21,162 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 03:02:21,162 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1966670640)] 03:02:21,164 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1966670640)] 03:02:21,171 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 03:02:21,171 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 03:02:21,171 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 03:02:21,171 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 03:02:21,171 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-24336561)] 03:02:21,171 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-24336561)] 03:02:21,184 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 03:02:21,184 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 03:02:21,184 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying 2 components for commit 03:02:21,184 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 03:02:21,184 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 03:02:21,184 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 03:02:21,184 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 03:02:21,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_4#-1966670640)] 03:02:21,184 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard people 03:02:21,184 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 03:02:21,184 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 03:02:21,184 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 03:02:21,184 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_5#-24336561)] 03:02:21,185 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,185 PM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,186 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 03:02:21,186 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying 1 components for commit 03:02:21,186 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying transaction for shard people 03:02:21,188 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 03:02:21,189 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 canCommit 03:02:21,190 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,190 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,190 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 13 03:02:21,190 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,190 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 03:02:21,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,190 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 03:02:21,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 13 03:02:21,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,191 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 03:02:21,191 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:02:21,194 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 03:02:21,194 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] resolved 03:02:21,194 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 03:02:21,194 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] resolved 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: successfully resolved all cohort actors 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finishCanCommit 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - previous Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 readied - proceeding to FindPrimaryShard 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 directCommit called 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 sending 1 batched modifications, ready: true 03:02:21,195 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 03:02:21,196 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 03:02:21,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 03:02:21,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:21,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 03:02:21,202 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:21,202 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 03:02:21,202 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 of 1 operations, client version 13 03:02:21,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 ready 03:02:21,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 03:02:21,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 is scheduled for canCommit step 03:02:21,205 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:21,205 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] 03:02:21,205 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] 03:02:21,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 03:02:21,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:21,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 03:02:21,210 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:21,210 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit returning result: true 03:02:21,210 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 commit 03:02:21,210 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finish commit 03:02:21,211 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] 03:02:21,211 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268)] 03:02:21,211 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] 03:02:21,211 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#-1090524550)] 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,217 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}]]] 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 03:02:21,218 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 224 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 validated 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: success 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 candidate ready 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - messages: [] 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 prepared 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 scheduled for commit step 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 221 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 03:02:21,219 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 03:02:21,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:21,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 03:02:21,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:02:21,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}]]] 03:02:21,220 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 submitted to persistence 03:02:21,220 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 03:02:21,220 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 03:02:21,220 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}]]] 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=235} 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 3 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 3 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 03:02:21,222 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:21,223 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1926803061] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 03:02:21,223 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 03:02:21,223 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1926803061] 03:02:21,223 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,223 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1926803061] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:21,224 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 96.80 μs 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}]]] 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=224} 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=221} 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 03:02:21,226 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:21,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 03:02:21,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 03:02:21,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 03:02:21,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 03:02:21,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 03:02:21,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 03:02:21,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:02:21,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:02:21,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%23285506268$r] 03:02:21,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 3 03:02:21,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:02:21,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 03:02:21,229 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,229 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 117.3 μs 03:02:21,230 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=235}] 03:02:21,230 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 03:02:21,230 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 03:02:21,230 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 03:02:21,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 03:02:21,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 03:02:21,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 03:02:21,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 03:02:21,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 03:02:21,238 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,239 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,240 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 3 03:02:21,240 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 03:02:21,240 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 185.9 μs 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%23-1090524550$s] 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 03:02:21,240 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards$p] 03:02:21,242 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 3 03:02:21,243 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:02:21,243 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 03:02:21,244 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,244 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 106.7 μs 03:02:21,244 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,245 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 45.94 μs 03:02:21,248 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 03:02:21,248 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit succeeded 03:02:21,248 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 03:02:21,248 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 03:02:21,248 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 03:02:21,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 03:02:21,249 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=13] 03:02:21,249 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:02:21,249 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 canCommit 03:02:21,250 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 successfully completed direct commit 03:02:21,250 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 03:02:21,250 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 03:02:21,250 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 03:02:21,250 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:02:21,250 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 3 03:02:21,251 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 03:02:21,251 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,251 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 53.51 μs 03:02:21,251 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:02:21,251 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 31.96 μs 03:02:21,251 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 03:02:21,252 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:21,252 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:21,252 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 03:02:21,252 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 03:02:21,252 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:02:21,255 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 03:02:21,255 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:21,255 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:21,256 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#91724342, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=13] 03:02:21,258 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#91724342] 03:02:21,263 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:21,263 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#91724342)] 03:02:21,268 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 03:02:21,268 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 03:02:21,268 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:21,269 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 03:02:21,269 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 03:02:21,269 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:02:21,270 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 03:02:21,273 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#1091409029] 03:02:21,274 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#1091409029, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=13] 03:02:21,274 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 03:02:21,274 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#1091409029)] 03:02:21,285 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 03:02:21,286 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:02:21,286 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:21,286 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 03:02:21,286 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 03:02:21,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 03:02:21,287 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 03:02:21,287 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 03:02:21,287 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 03:02:21,291 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:02:21,291 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:21,291 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:21,291 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:02:21,291 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5bdb6842 03:02:21,291 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5bdb6842 03:02:21,292 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 03:02:21,292 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:02:21,293 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:21,293 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:21,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50354fbe 03:02:21,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@50354fbe 03:02:21,293 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 03:02:21,294 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:02:21,301 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards] 03:02:21,301 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards] 03:02:21,301 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:02:21,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:21,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:21,301 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:02:21,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:21,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:21,305 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionChainWithMultipleShards, matchIndex: 4, lastIndex: 4, isVoting: true 03:02:21,305 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:02:21,305 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:02:21,305 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 03:02:21,306 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithMultipleShards, matchIndex: 3, lastIndex: 3, isVoting: true 03:02:21,306 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:02:21,306 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:02:21,306 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 03:02:21,527 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 213, lastApplied : 4, commitIndex : 4 03:02:21,702 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 03:02:21,702 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] ShardManager - shard-manager-testTransactionChainWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#285506268]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:02:21,702 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 03:02:21,702 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-139: Closing 03:02:21,702 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 03:02:21,703 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-138: Closing 03:02:21,703 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 03:02:21,703 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 03:02:21,703 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 03:02:21,703 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 03:02:21,705 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 03:02:21,705 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 03:02:21,705 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:02:21,706 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 03:02:21,707 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:21,707 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:21,723 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:21,723 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:21,723 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:21,724 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:21,730 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:21,743 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 03:02:21,744 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 03:02:21,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 03:02:21,744 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 03:02:21,744 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 03:02:21,744 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 03:02:21,744 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 03:02:21,745 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 03:02:21,750 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:21,751 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:22,731 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2023494382064820608] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:22,764 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:22,764 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:22,766 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:22.768] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:22,773 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [790739844027828250] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:22,776 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:22,776 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:22,780 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:22.782] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:22,803 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:02:22,815 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-585924863079122726] 03:02:22,816 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:22,821 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:22,821 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:22,823 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:22,823 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:22,835 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:22,837 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:22,837 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:22,837 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:22,848 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1481443039886243873] 03:02:22,849 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:22,852 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:22,852 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:22,853 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:22,853 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:22,884 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:22,921 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 03:02:22,921 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_7#1091409029)] 03:02:22,921 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 03:02:22,922 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_6#91724342)] 03:02:22,928 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1750694324924267513] 03:02:22,928 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:22,938 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:22,943 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:22,943 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:22,943 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:22,957 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:22,969 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:22,973 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 03:02:22,986 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:22,999 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:23,4 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 03:02:23,8 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:23,19 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 03:02:23,19 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 03:02:23,20 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:23,20 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,22 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 03:02:23,22 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 03:02:23,22 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7f20abe7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a364b54 03:02:23,28 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6f9b16e6 03:02:23,28 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1709823743022, None)) 03:02:23,28 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823743021,0,0)) 03:02:23,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a364b54 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@778569f2 03:02:23,29 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@35461a16 03:02:23,29 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 03:02:23,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:23,30 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 03:02:23,30 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:23,33 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:23,37 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 03:02:23,41 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,41 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,41 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1144751330] 03:02:23,42 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 03:02:23,45 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 03:02:23,45 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1144751330] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:23,46 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#1333975733 created and ready for shard:member-1-shard-cars-testInstallSnapshot 03:02:23,46 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#1333975733] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:23,46 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#1773205784] 03:02:23,46 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#1773205784] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:23,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#1941621925] 03:02:23,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 03:02:23,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 03:02:23,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 03:02:23,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 03:02:23,56 PM [cluster-test-shard-dispatcher-15] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 03:02:23,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1054958880] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 03:02:23,57 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 929.2 μs 03:02:23,57 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 1.146 ms: snapshotIndex=5, snapshotTerm=1, journal-size=0 03:02:23,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1054958880] 03:02:23,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 03:02:23,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 89.50 μs 03:02:23,58 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:23,59 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:23,59 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 2.599 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 03:02:23,62 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#1054958880] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:23,63 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:23,63 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 03:02:23,63 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 03:02:23,81 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 03:02:23,81 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 03:02:23,81 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 03:02:23,84 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 03:02:23,84 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 03:02:23,84 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 03:02:23,85 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1862365543] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:23,91 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 03:02:23,91 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 03:02:23,91 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3ec8bb4a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5d10eecd 03:02:23,92 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 03:02:23,92 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@f3ca227 03:02:23,92 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 03:02:23,92 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:23,92 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,93 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1709823743093, None)) 03:02:23,93 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823743092,0,0)) 03:02:23,94 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5d10eecd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2ce6d74e 03:02:23,94 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3dd1f214 03:02:23,94 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 03:02:23,95 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:23,95 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 03:02:23,96 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,96 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:02:23,96 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-2-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#911821274] 03:02:23,97 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 03:02:23,97 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#911821274] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:23,97 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 03:02:23,97 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-485243530 created and ready for shard:member-2-shard-cars-testInstallSnapshot 03:02:23,98 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-485243530] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:23,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:23,99 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:23,99 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 03:02:23,99 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 03:02:23,99 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 03:02:23,100 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 03:02:23,100 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,100 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 03:02:23,100 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 03:02:23,100 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 03:02:23,102 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#-955117164] 03:02:23,102 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#-955117164] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:23,144 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] 03:02:23,145 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 03:02:23,145 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1144751330] 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 03:02:23,398 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 03:02:23,399 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 03:02:23,399 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 03:02:23,692 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:23,708 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 03:02:23,708 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 03:02:23,708 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 03:02:23,840 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:23,840 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:23,841 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:23,841 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 03:02:23,841 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,842 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:23,842 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:02:23,842 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,842 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:02:23,860 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:23,860 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 03:02:23,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:02:23,860 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:23,860 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:02:23,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:02:23,962 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:23,975 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 03:02:23,975 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 03:02:23,975 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 03:02:23,975 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 03:02:23,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:23,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 03:02:23,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 03:02:23,979 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 03:02:23,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=null 03:02:23,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:23,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 03:02:23,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:23,981 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:23,981 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:23,981 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 03:02:23,981 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 03:02:23,981 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:23,981 PM [cluster-test-shard-dispatcher-15] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 03:02:23,981 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 03:02:23,981 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 03:02:23,982 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 03:02:23,982 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:23,982 PM [cluster-test-shard-dispatcher-15] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:23,982 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 03:02:23,982 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 03:02:23,985 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 03:02:23,985 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:02:23,985 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 03:02:23,985 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 03:02:23,985 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:23,985 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:23,985 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 03:02:23,986 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 03:02:23,986 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 03:02:23,986 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 03:02:23,986 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 03:02:23,990 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 03:02:23,990 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 3 03:02:23,990 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 03:02:23,992 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 03:02:23,992 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@36f8ddae, electionTerm=4, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 03:02:23,992 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 03:02:23,993 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 03:02:23,993 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 03:02:23,993 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 03:02:23,993 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 03:02:23,993 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 3 03:02:23,993 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 03:02:23,996 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 03:02:23,996 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 03:02:23,997 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 03:02:24,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=4, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 03:02:24,6 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 03:02:24,6 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 03:02:24,7 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@4aa907bc, electionTerm=4, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 03:02:24,7 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1087433538] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 463.3 μs 03:02:24,8 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 03:02:24,10 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1087433538] 03:02:24,10 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 03:02:24,10 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 03:02:24,10 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 153.5 μs 03:02:24,10 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] 03:02:24,10 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:24,10 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1087433538] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:24,10 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 Queuing TransactionOperation 03:02:24,11 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1144751330, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4300452e] 03:02:24,11 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1144751330, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4300452e] 03:02:24,11 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1144751330)] for shard cars 03:02:24,11 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:02:24,12 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3e32d1c0 03:02:24,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=4, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 03:02:24,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 03:02:24,16 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 03:02:24,16 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 03:02:24,16 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 03:02:24,20 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 03:02:24,20 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 03:02:24,20 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 03:02:24,20 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 03:02:24,20 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@608ce9ff 03:02:24,20 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@608ce9ff 03:02:24,20 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 03:02:24,22 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 03:02:24,22 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:24,22 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:24,25 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=4, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 03:02:24,25 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 03:02:24,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testInstallSnapshot] 03:02:24,26 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 03:02:24,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:24,26 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:24,223 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testInstallSnapshot, matchIndex: 5, lastIndex: 5, isVoting: true 03:02:24,223 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Follower's log matches - sending ElectionTimeout 03:02:24,223 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 03:02:24,223 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leader transfer complete 03:02:24,439 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 03:02:24,439 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] ShardManager - shard-manager-testInstallSnapshot: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-1144751330]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:02:24,440 PM [cluster-test-shard-dispatcher-15] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 03:02:24,440 PM [cluster-test-shard-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-144: Closing 03:02:24,440 PM [cluster-test-shard-dispatcher-15] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 03:02:24,440 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 03:02:24,441 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 03:02:24,441 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 03:02:24,441 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: shutting down 03:02:24,441 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 03:02:24,441 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 03:02:24,442 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:24,442 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:24,442 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:24,442 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-256415227] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:24,442 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#1912802772] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1912802772] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:24,443 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:24,444 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:24,455 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:24,456 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:24,457 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:24,457 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:24,464 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:24,492 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 03:02:24,492 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 03:02:24,492 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 03:02:24,492 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 03:02:24,493 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 03:02:24,494 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:24,494 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:25,459 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-585924863079122726] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:25,509 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1481443039886243873] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/07/2024 15:02:25.518] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-80)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:02:25.518] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-80)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:02:25.522] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:25,545 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:25,545 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:02:25.552] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:25,567 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:25,578 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7033741788278287005] 03:02:25,579 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:25,580 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:25,580 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:25,581 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:25,581 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:25,593 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:25,593 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:25,593 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:25,594 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:25,602 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7709695644995867209] 03:02:25,602 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:25,605 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:25,605 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:25,607 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:25,607 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:25,622 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:25,637 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5585634526086970625] 03:02:25,637 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:25,649 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:25,649 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:25,650 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:25,650 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:25,652 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:25,654 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:25,657 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 03:02:25,688 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:02:25,693 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#702093224] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:25,699 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 03:02:25,699 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:02:25,700 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5507afda to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@25cca69c 03:02:25,700 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@64981a69 03:02:25,700 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 03:02:25,700 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,700 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:25,700 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,701 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709823745701, None)) 03:02:25,701 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823745700,0,0)) 03:02:25,702 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2cc97b85 03:02:25,702 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@25cca69c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@538601f3 03:02:25,702 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 03:02:25,702 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:25,704 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:25,704 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,705 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:25,707 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,707 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,707 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] 03:02:25,707 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 03:02:25,708 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 03:02:25,709 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 03:02:25,711 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:25,711 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#301456033] 03:02:25,711 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-544657421 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,711 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-544657421] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:25,711 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#301456033] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:25,719 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:25,719 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:25,719 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:25,719 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 03:02:25,719 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 03:02:25,719 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 03:02:25,720 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 03:02:25,722 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:25,722 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 03:02:25,732 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:25,733 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:25,733 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-449343491] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:25,733 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 03:02:25,737 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:25,740 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:02:25,746 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 03:02:25,746 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:02:25,746 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@26cd543b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@724480b9 03:02:25,746 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@17a1d0af 03:02:25,746 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709823745746, None)) 03:02:25,746 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 03:02:25,746 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823745745,0,0)) 03:02:25,746 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,747 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@724480b9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@46e2ab74 03:02:25,747 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:25,747 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@72bb760e 03:02:25,747 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 03:02:25,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:25,747 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,748 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,748 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,748 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1655056523] 03:02:25,748 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 03:02:25,749 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 03:02:25,755 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1655056523] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:25,755 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#100508105] 03:02:25,755 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#819762324 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,756 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#100508105] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:25,756 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#819762324] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:25,762 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:25,762 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:25,762 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 03:02:25,762 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 03:02:25,762 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:25,762 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 03:02:25,762 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:25,763 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 03:02:25,763 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 03:02:25,763 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 03:02:25,798 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#702093224] 03:02:25,798 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 03:02:25,798 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] 03:02:26,3 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 03:02:26,4 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 03:02:26,4 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 03:02:26,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 03:02:26,303 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,303 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 03:02:26,332 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:26,594 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:26,594 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:26,595 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:26,595 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,595 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,595 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:26,595 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,595 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,595 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,595 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,603 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 03:02:26,603 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,603 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 03:02:26,603 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,603 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,607 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:26,608 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 03:02:26,608 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:26,608 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:26,608 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=null 03:02:26,608 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:26,608 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,608 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:02:26,611 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 03:02:26,612 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 03:02:26,612 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 03:02:26,614 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:26,616 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 03:02:26,616 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 03:02:26,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 03:02:26,617 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:26,617 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:26,617 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 03:02:26,618 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 03:02:26,618 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:26,655 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:26,655 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,655 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,656 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 03:02:26,656 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:26,656 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,656 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,656 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 03:02:26,656 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:26,656 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:26,660 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:26,663 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 03:02:26,668 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#710097706] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:26,668 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:26,671 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 03:02:26,671 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:02:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6fe39e3f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@462bf72 03:02:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5ec6754e 03:02:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709823746672, None)) 03:02:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823746671,0,0)) 03:02:26,672 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@462bf72 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@fcc6170 03:02:26,673 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 03:02:26,674 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:02:26,675 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:02:26,675 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:02:26,676 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:26,676 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:26,684 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:26,684 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:26,684 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:26,684 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:26,686 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:26,702 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#702093224] 03:02:26,702 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 03:02:26,702 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] 03:02:26,702 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] 03:02:26,728 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 03:02:27,613 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:02:27,694 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [5585634526086970625] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:27,695 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [5585634526086970625] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:32,720 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 03:02:32,722 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:32,723 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:32,761 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:32,844 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-449343491] 03:02:32,844 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 03:02:32,844 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1655056523] 03:02:32,844 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] 03:02:32,844 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=4, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=4, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=4, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=6, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=6, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:32,845 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 03:02:32,845 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=6, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=6, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 03:02:32,845 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 03:02:32,845 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:32,845 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1655056523] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:32,845 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#1655056523] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:32,846 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441] 03:02:32,847 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:02:32,848 PM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:02:32,848 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:02:32,849 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:02:32,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:32,849 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 03:02:32,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 03:02:32,849 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 03:02:32,850 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12f30f7b 03:02:32,850 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@12f30f7b 03:02:32,850 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 03:02:32,850 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:32,850 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:32,852 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 03:02:32,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:32,852 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:32,853 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader] 03:02:32,853 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 03:02:32,853 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:32,853 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:32,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, matchIndex: -1, lastIndex: -1, isVoting: true 03:02:32,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Follower's log matches - sending ElectionTimeout 03:02:32,854 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:32,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:32,855 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leader transfer complete 03:02:33,263 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 03:02:33,264 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#239624441]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:02:33,264 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:33,264 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-149: Closing 03:02:33,264 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 03:02:33,265 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:02:33,265 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:02:33,265 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:02:33,265 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 03:02:33,265 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:02:33,265 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:02:33,266 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#702093224] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#702093224] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:33,266 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#702093224] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#702093224] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:33,266 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-1479317169] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1479317169] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:33,266 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-1479317169] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1479317169] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:33,268 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:33,268 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:33,743 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 03:02:33,743 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 03:02:33,764 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 03:02:34,282 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7033741788278287005] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:34,287 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:34,287 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:34,290 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:34.293] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:34,315 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:02:34,316 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:02:34,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 03:02:34,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 03:02:34,317 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:02:34,317 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:34,317 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:34,324 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:34.325] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:34,350 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:02:34,360 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3273244626485709139] 03:02:34,360 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:34,363 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:34,363 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:34,365 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:34,365 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:34,378 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:34,384 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:34,384 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:34,384 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:34,395 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4304939406268927060] 03:02:34,396 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:34,398 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:34,398 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:34,403 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:34,413 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:34,426 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:02:34,444 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2730233012427331132] 03:02:34,445 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:34,460 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:34,462 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:34,464 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 03:02:34,465 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:34,465 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:34,465 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:34,465 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:34,482 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:34,486 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 03:02:34,489 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 03:02:34,489 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:02:34,489 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2872f011 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@f9878e4 03:02:34,490 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@19905d97 03:02:34,490 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1709823754489, None)) 03:02:34,490 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823754488,0,0)) 03:02:34,490 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@f9878e4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@68866223 03:02:34,490 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 03:02:34,491 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:34,494 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:34,497 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 03:02:34,498 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 03:02:34,498 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,499 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:34,499 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@484bb08f 03:02:34,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:34,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,499 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,499 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577] 03:02:34,500 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 03:02:34,500 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:34,501 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 03:02:34,502 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1253858343 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,502 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1253858343] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:34,522 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:34,522 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:34,522 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:34,522 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:34,522 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 03:02:34,522 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 03:02:34,522 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 03:02:34,523 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 03:02:34,523 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 03:02:34,525 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 03:02:34,526 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-148383698] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:34,529 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:34,541 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 03:02:34,544 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#1479915246] 03:02:34,544 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#1479915246] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:34,548 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 03:02:34,548 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,548 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:34,548 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,549 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 03:02:34,549 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:02:34,549 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@a6a0f47 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4adf07a 03:02:34,549 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@926437 03:02:34,556 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1709823754555, None)) 03:02:34,556 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823754554,0,0)) 03:02:34,556 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4adf07a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@741fd822 03:02:34,556 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1b06f5af 03:02:34,558 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 03:02:34,558 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:34,558 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,559 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,559 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,559 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1521810651] 03:02:34,560 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 03:02:34,560 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 03:02:34,560 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1521810651] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:34,560 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1615214134] 03:02:34,561 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-181869197] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:34,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:34,563 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:34,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:34,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 03:02:34,563 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 03:02:34,563 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 03:02:34,570 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:34,576 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:34,578 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-181869197 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,578 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 03:02:34,578 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 03:02:34,578 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 03:02:34,582 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1615214134] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:34,609 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] 03:02:34,609 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 03:02:34,609 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577] 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:34,782 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 03:02:34,782 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 03:02:34,783 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 03:02:35,81 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:02:35,82 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,82 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 03:02:35,106 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:35,352 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:02:35,352 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,352 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 03:02:35,384 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:35,384 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:35,384 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:35,384 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,384 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,384 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:35,384 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,384 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,385 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,399 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:35,399 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,399 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,399 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:35,399 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,399 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,472 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:35,621 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:02:35,621 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,622 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 03:02:35,622 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,627 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:35,627 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 03:02:35,627 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:35,627 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:35,627 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 03:02:35,627 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:35,627 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,627 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:35,634 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:35,634 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:35,634 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 03:02:35,634 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:35,635 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:35,635 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:02:35,635 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 03:02:35,635 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 03:02:35,635 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 03:02:35,635 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:35,635 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:35,635 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 03:02:35,635 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 03:02:35,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 03:02:35,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,636 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 03:02:35,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:35,636 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:35,636 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 03:02:35,636 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 03:02:35,639 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 03:02:35,640 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:35,717 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-148383698] 03:02:35,717 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 03:02:35,717 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1521810651] 03:02:35,718 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,718 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-148383698] 03:02:35,718 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 03:02:35,718 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-1521810651] 03:02:35,730 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,730 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,730 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 03:02:35,734 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 03:02:35,734 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:02:35,734 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:02:35,734 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:35,734 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:35,734 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,734 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 03:02:35,734 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-97-0 of 1 operations, client version 13 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-97-0 ready 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 validated 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-97-0: success 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 candidate ready 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-97-0: canCommit - candidate rootPath: / 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-97-0: canCommit - messages: [] 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-97-0: preCommit - successfulFromPrevious: [] 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 prepared 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,735 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,736 PM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 03:02:35,736 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:02:35,736 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:02:35,736 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-97-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:35,736 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:35,736 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,736 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 submitted to persistence 03:02:35,737 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:02:35,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 03:02:35,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:02:35,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:35,737 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:02:35,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:02:35,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:02:35,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 03:02:35,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:35,738 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:02:35,738 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=5228470} is already disabled 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 03:02:35,739 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 03:02:35,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:02:35,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 03:02:35,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 03:02:35,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:02:35,741 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-97-0: commit - successfulFromPrevious: [] 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 changing state from COMMIT_PENDING to COMMITTED 03:02:35,742 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-211#1356756856] 03:02:35,743 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#236271662] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 03:02:35,743 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#236271662] 03:02:35,743 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 03:02:35,743 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 03:02:35,743 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:35,743 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#236271662] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:35,743 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 03:02:35,744 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 132.7 μs 03:02:35,745 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 03:02:35,745 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] 03:02:35,745 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 03:02:35,745 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 03:02:35,745 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:35,745 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 03:02:35,745 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 03:02:35,745 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 03:02:35,745 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-97-0 03:02:35,746 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:35,746 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6c45c5c8] 03:02:35,746 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#136637010] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 03:02:35,746 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 03:02:35,746 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 03:02:35,746 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#136637010] 03:02:35,747 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 03:02:35,747 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 101.2 μs 03:02:35,747 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#136637010] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:35,747 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6c45c5c8] 03:02:35,747 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577)] for shard cars 03:02:35,747 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:02:35,747 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@752c2e9d 03:02:35,748 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,748 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,748 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 03:02:35,750 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,750 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-98-0 of 1 operations, client version 13 03:02:35,750 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,750 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-98-0 ready 03:02:35,754 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0 canCommit 03:02:35,754 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577)] resolved 03:02:35,754 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: successfully resolved all cohort actors 03:02:35,754 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0 finishCanCommit 03:02:35,754 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-98-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577)] 03:02:35,754 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577)] 03:02:35,754 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 validated 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-98-0: success 03:02:35,755 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:35,755 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: canCommit returning result: true 03:02:35,755 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0 commit 03:02:35,755 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0 finish commit 03:02:35,755 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-98-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577)] 03:02:35,755 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577)] 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 candidate ready 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-98-0: canCommit - candidate rootPath: / 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-98-0: canCommit - messages: [] 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-98-0: preCommit - successfulFromPrevious: [] 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 prepared 03:02:35,755 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-98-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 03:02:35,756 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 submitted to persistence 03:02:35,757 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:02:35,758 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 03:02:35,758 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:02:35,758 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:02:35,758 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 03:02:35,758 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:02:35,758 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 03:02:35,758 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:35,759 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:02:35,759 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 03:02:35,759 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:35,759 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 03:02:35,759 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 03:02:35,759 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 03:02:35,759 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,759 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-98-0: commit - successfulFromPrevious: [] 03:02:35,759 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 changing state from COMMIT_PENDING to COMMITTED 03:02:35,760 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%23-957964577$a] 03:02:35,760 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: commit succeeded 03:02:35,760 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 03:02:35,760 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 03:02:35,760 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 116.9 μs 03:02:35,760 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:35,760 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 03:02:35,760 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 03:02:35,760 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:35,760 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@752c2e9d 03:02:35,760 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:35,761 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:02:35,761 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:35,761 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 03:02:35,761 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 03:02:35,761 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 03:02:35,761 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 03:02:35,761 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2432e770 03:02:35,761 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2432e770 03:02:35,761 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:35,761 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:35,761 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 03:02:35,761 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:35,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 03:02:35,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:02:35,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 03:02:35,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 03:02:35,763 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 03:02:35,764 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-98-0 03:02:35,765 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 03:02:35,765 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 03:02:35,765 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 03:02:35,765 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 80.81 μs 03:02:35,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader] 03:02:35,766 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 03:02:35,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:35,766 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:35,768 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 03:02:35,768 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 03:02:35,768 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:35,768 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leader transfer complete 03:02:35,995 PM [cluster-test-shard-dispatcher-32] [WARN] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 222, lastApplied : 2, commitIndex : 2 03:02:36,172 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 03:02:36,172 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-957964577]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:02:36,173 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:36,173 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-155: Closing 03:02:36,173 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 03:02:36,173 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 03:02:36,174 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 03:02:36,174 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:02:36,174 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 03:02:36,174 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 03:02:36,174 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 03:02:36,174 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:36,175 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-788795020] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-788795020] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:36,175 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:36,175 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1026440858] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:36,175 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-788795020] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-788795020] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:36,175 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:36,175 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:36,183 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:36,183 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:36,184 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:36,184 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:36,186 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:36,200 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:02:36,200 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 03:02:36,200 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 03:02:36,200 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 03:02:36,201 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 03:02:36,202 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:36,202 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:37,192 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3273244626485709139] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:37,219 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:37,220 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:37,223 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4304939406268927060] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:37,223 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:37.226] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:37,244 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:37,244 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:37,253 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:37.255] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:37,289 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:37,298 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8453867941489604487] 03:02:37,298 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:37,306 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:37,306 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:37,311 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:37,311 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:37,312 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:37,312 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:37,312 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:37,320 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:37,334 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-731124023012606942] 03:02:37,334 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:37,348 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:37,348 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:37,348 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:37,348 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:37,390 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:02:37,403 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2476026078114837941] 03:02:37,404 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:37,405 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:37,406 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:37,414 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:37,415 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:37,418 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:37,420 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:37,422 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 03:02:37,424 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:37,424 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:37,426 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-530922633] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:37,426 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 03:02:37,441 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 03:02:37,441 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:02:37,441 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@67722af to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@212336c3 03:02:37,441 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@37bba4fb 03:02:37,441 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 03:02:37,441 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1709823757441, None)) 03:02:37,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,441 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823757440,0,0)) 03:02:37,441 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@212336c3 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6658dde0 03:02:37,441 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:37,441 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 03:02:37,441 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1d5465e6 03:02:37,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:37,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,442 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:37,444 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:37,446 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 03:02:37,453 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,453 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-504166952] 03:02:37,453 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 03:02:37,454 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 03:02:37,455 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-504166952] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:37,455 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-1323055923 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,456 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-1323055923] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:37,456 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#548865553] 03:02:37,456 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#548865553] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:37,464 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1326499700] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:37,466 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 03:02:37,469 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:37,469 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:37,469 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:37,469 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 03:02:37,469 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 03:02:37,469 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 03:02:37,470 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 03:02:37,470 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 03:02:37,482 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 03:02:37,490 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:37,492 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:37,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 03:02:37,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:02:37,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@52102f8b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1972ac94 03:02:37,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@63081aaf 03:02:37,494 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1709823757494, None)) 03:02:37,494 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823757493,0,0)) 03:02:37,495 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1972ac94 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@42874c97 03:02:37,496 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 03:02:37,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,497 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:37,497 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,497 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@44606242 03:02:37,505 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 03:02:37,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:37,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,506 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,506 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,506 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-2-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#1559559755] 03:02:37,507 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 03:02:37,508 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 03:02:37,508 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#1559559755] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:37,508 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#597302166] 03:02:37,508 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-940932625 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,509 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-940932625] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:37,509 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#597302166] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:37,514 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:37,514 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:37,514 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:37,514 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 03:02:37,514 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 03:02:37,515 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 03:02:37,517 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 03:02:37,517 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 03:02:37,517 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 03:02:37,556 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-530922633] 03:02:37,556 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 03:02:37,556 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-504166952] 03:02:37,759 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:37,759 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 03:02:37,759 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 03:02:37,759 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:37,760 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 03:02:37,760 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:37,760 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 03:02:37,760 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:37,760 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 03:02:37,760 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 03:02:37,760 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 03:02:38,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:02:38,9 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,9 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 03:02:38,74 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:38,260 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:02:38,260 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,260 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 03:02:38,331 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:38,331 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:38,332 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:38,332 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,332 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,333 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:38,333 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,333 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,333 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,347 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:38,347 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,347 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,347 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:38,347 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,347 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,440 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:38,529 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:02:38,529 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,529 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 03:02:38,529 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:38,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 03:02:38,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:38,532 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:38,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 03:02:38,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:38,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,532 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:38,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:38,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:38,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 03:02:38,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:38,534 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:38,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:02:38,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 03:02:38,536 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 03:02:38,536 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:02:38,536 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:38,536 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:38,536 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 03:02:38,536 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 03:02:38,537 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 03:02:38,537 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,537 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:02:38,537 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:38,537 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:38,537 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 03:02:38,537 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 03:02:38,538 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 03:02:38,538 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:38,565 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testReadWriteTransactionWithSingleShard commit rateLimit adjusted to 22.90697868818152 03:02:38,565 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,565 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1326499700] 03:02:38,566 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:38,566 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:38,566 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:38,566 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:38,566 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:38,566 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 03:02:38,566 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 03:02:38,567 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars 03:02:38,567 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 03:02:38,567 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 03:02:38,569 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 03:02:38,569 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 03:02:38,569 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 03:02:38,569 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:38,569 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:38,569 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,569 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 03:02:38,569 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,570 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#1580199163] 03:02:38,570 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#1580199163] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:38,571 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 03:02:38,572 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#1580199163, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, version=13] 03:02:38,572 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:38,572 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:38,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 03:02:38,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:02:38,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:38,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 03:02:38,572 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:02:38,573 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 03:02:38,573 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:38,573 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 3 batched modifications, ready: false 03:02:38,573 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#1580199163)] 03:02:38,573 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 03:02:38,573 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:38,573 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#1580199163)] 03:02:38,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 03:02:38,575 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=7355293} is already disabled 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:38,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 03:02:38,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 03:02:38,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:02:38,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 03:02:38,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:02:38,583 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 03:02:38,583 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:02:38,583 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 03:02:38,583 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 03:02:38,583 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=3] 03:02:38,583 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 03:02:38,584 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 03:02:38,584 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 03:02:38,584 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 1 batched modifications, ready: false 03:02:38,584 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#1580199163)] 03:02:38,584 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#1580199163)] 03:02:38,589 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 DataExists operation succeeded 03:02:38,589 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=1] 03:02:38,589 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 03:02:38,589 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 03:02:38,589 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 03:02:38,589 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 0 batched modifications, ready: true 03:02:38,589 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_8#1580199163)] 03:02:38,590 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 03:02:38,591 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,591 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,591 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, client version 13 03:02:38,591 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: canCommit for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: success 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,592 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,593 PM [cluster-test-shard-dispatcher-26] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 249 03:02:38,593 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 03:02:38,593 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 03:02:38,593 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:38,593 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:38,593 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,594 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 03:02:38,594 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 03:02:38,595 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 03:02:38,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 03:02:38,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:02:38,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:38,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=249} 03:02:38,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 03:02:38,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 03:02:38,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:38,597 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithSingleShard_member-1-shard-cars-testReadWriteTransactionWithSingleShard_shard-cars-member-2%3Adatastore-testReadWriteTransactionWithSingleShard%400%3A0_8%231580199163$f] 03:02:38,597 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1227575130] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 03:02:38,597 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1227575130] 03:02:38,598 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:02:38,598 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:02:38,598 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:38,598 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 116.8 μs 03:02:38,598 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 03:02:38,598 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1227575130] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:38,601 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 03:02:38,601 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 03:02:38,601 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 03:02:38,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 03:02:38,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:38,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 03:02:38,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:02:38,601 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 03:02:38,601 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:38,602 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:38,602 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 03:02:38,602 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 03:02:38,602 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 03:02:38,602 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:38,602 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1726473136] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 03:02:38,603 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:02:38,603 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 03:02:38,603 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1726473136] 03:02:38,604 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:02:38,604 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1726473136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:38,604 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 142.6 μs 03:02:38,605 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 03:02:38,606 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-89298082] 03:02:38,606 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-89298082] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:38,607 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-89298082, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0, version=13] 03:02:38,607 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:38,607 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-89298082)] 03:02:38,610 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 03:02:38,610 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:02:38,610 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:38,610 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 03:02:38,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 03:02:38,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 03:02:38,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5e55002 03:02:38,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5e55002 03:02:38,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 03:02:38,610 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:38,611 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:02:38,611 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:38,611 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:38,612 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard] 03:02:38,612 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:02:38,612 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:38,612 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:38,613 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 03:02:38,613 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 03:02:38,613 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:38,613 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leader transfer complete 03:02:39,29 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 03:02:39,30 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-504166952]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:02:39,30 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:02:39,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-160: Closing 03:02:39,30 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:02:39,30 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 03:02:39,31 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 03:02:39,31 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:02:39,31 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 03:02:39,31 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 03:02:39,31 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 03:02:39,33 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:39,33 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:39,40 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:39,41 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:39,41 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:39,41 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:39,52 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 03:02:39,52 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:02:39,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 03:02:39,52 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:02:39,53 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 03:02:39,54 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:39,54 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:40,49 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-8453867941489604487] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:40,69 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:40,69 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:40,70 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-731124023012606942] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/07/2024 15:02:40.077] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:40,88 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:40,88 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:40,93 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:40.096] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:40,114 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:02:40,127 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8977649116481891395] 03:02:40,127 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:40,129 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:40,129 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:40,130 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:40,130 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:40,151 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:40,152 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:40,152 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:40,152 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:40,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [115921182919951560] 03:02:40,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:40,168 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:40,169 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:40,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:40,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:40,197 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:40,213 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7720650371098035102] 03:02:40,213 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:40,228 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:40,228 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:40,228 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:40,228 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:40,230 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:40,241 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:40,245 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 03:02:40,246 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 03:02:40,246 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_9#-89298082)] 03:02:40,268 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:40,271 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:40,288 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1864511985] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,289 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 03:02:40,300 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 03:02:40,300 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:02:40,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 03:02:40,300 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@357e0f29 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3bb9e7f6 03:02:40,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,300 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3f866459 03:02:40,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:40,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,300 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,301 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1709823760301, None)) 03:02:40,303 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:40,307 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823760300,0,0)) 03:02:40,308 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3bb9e7f6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4bb474aa 03:02:40,308 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 03:02:40,308 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:40,309 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:40,311 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6b866595 03:02:40,311 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:40,311 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,311 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,311 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,311 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] 03:02:40,311 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,312 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,312 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,312 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824] 03:02:40,312 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 03:02:40,312 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:40,312 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,313 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 03:02:40,313 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 03:02:40,314 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 03:02:40,314 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#1678782919 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,314 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#904355452] 03:02:40,314 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,314 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#1678782919] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,314 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#904355452] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,314 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#-289040066 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,316 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 03:02:40,317 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1769250101] 03:02:40,317 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#-289040066] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,318 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1769250101] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,323 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 03:02:40,325 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:40,325 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:40,325 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:40,325 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:40,325 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:02:40,325 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 03:02:40,325 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 03:02:40,325 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 03:02:40,325 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 03:02:40,325 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 03:02:40,326 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 03:02:40,327 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#957403272] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,328 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:02:40,328 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 03:02:40,328 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 03:02:40,328 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 03:02:40,328 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 03:02:40,337 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 03:02:40,337 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 03:02:40,349 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 03:02:40,349 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:02:40,349 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1d85aefb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1342433b 03:02:40,349 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@177dc58b 03:02:40,351 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 03:02:40,351 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:40,351 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 03:02:40,351 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:40,351 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,351 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,352 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1709823760352, None)) 03:02:40,352 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823760351,0,0)) 03:02:40,353 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1342433b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1ea090be 03:02:40,353 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2132cd46 03:02:40,354 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 03:02:40,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:40,355 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:40,356 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:40,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-2-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:40,356 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 03:02:40,357 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 03:02:40,357 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,358 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#330369689 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:40,358 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 03:02:40,358 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#330369689] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,358 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#2019620025] 03:02:40,358 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#2019620025] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,359 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 03:02:40,359 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-924813877 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 03:02:40,359 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-926509076] 03:02:40,361 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:02:40,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-2-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-429929985] 03:02:40,362 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-429929985] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,362 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-924813877] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,362 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-926509076] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:40,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:40,365 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:40,365 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 03:02:40,365 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 03:02:40,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:02:40,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 03:02:40,366 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,366 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 03:02:40,366 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 03:02:40,366 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 03:02:40,366 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 03:02:40,367 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:40,367 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:40,367 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 03:02:40,367 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 03:02:40,367 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:02:40,367 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 03:02:40,367 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 03:02:40,367 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,368 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 03:02:40,368 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 03:02:40,405 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1864511985] 03:02:40,405 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 03:02:40,405 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] 03:02:40,556 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:40,556 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 03:02:40,556 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 03:02:40,556 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,556 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 03:02:40,557 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:40,557 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 03:02:40,557 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:02:40,557 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 03:02:40,557 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 03:02:40,557 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 03:02:40,587 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 03:02:40,587 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 03:02:40,867 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:02:40,867 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:40,867 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 03:02:40,876 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:02:40,876 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:02:40,876 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 03:02:40,937 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:40,947 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:41,160 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:02:41,160 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:41,160 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 03:02:41,163 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:41,163 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,164 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,174 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,187 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:02:41,187 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:02:41,187 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 03:02:41,187 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:41,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 03:02:41,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:41,192 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:41,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=null 03:02:41,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:41,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:02:41,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:02:41,198 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:02:41,198 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:02:41,198 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 03:02:41,198 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:41,198 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:02:41,198 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 03:02:41,198 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 03:02:41,198 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 03:02:41,198 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 03:02:41,199 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 03:02:41,199 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 03:02:41,199 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 03:02:41,205 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 03:02:41,205 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:02:41,205 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 03:02:41,205 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 03:02:41,205 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 03:02:41,205 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 03:02:41,209 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 03:02:41,209 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:41,234 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:41,319 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 03:02:41,447 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:02:41,447 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:41,447 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 03:02:41,447 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:41,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 03:02:41,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:41,449 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:41,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 03:02:41,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:41,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:41,449 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:41,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:41,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 03:02:41,451 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 03:02:41,451 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 03:02:41,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 03:02:41,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:41,452 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 03:02:41,452 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 03:02:41,453 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:41,453 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:41,453 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 03:02:41,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 03:02:41,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:41,516 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1864511985] 03:02:41,517 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 03:02:41,517 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824] 03:02:41,519 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSnapshotOnRootOverwrite commit rateLimit adjusted to 85.79685415538616 03:02:41,519 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,524 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1864511985] 03:02:41,524 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1864511985] 03:02:41,524 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:41,524 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@30684e78] 03:02:41,524 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 03:02:41,524 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@419949bc] 03:02:41,524 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@30684e78] 03:02:41,525 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars 03:02:41,525 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:02:41,525 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,525 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@419949bc] 03:02:41,525 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] for shard default 03:02:41,525 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Shard default resolved to local data tree - adding local factory 03:02:41,525 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@10834be 03:02:41,527 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying 2 components for commit 03:02:41,527 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard cars 03:02:41,527 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 03:02:41,528 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,528 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,529 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 03:02:41,529 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,529 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,529 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:41,529 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:41,529 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,529 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard default 03:02:41,529 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 03:02:41,529 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:41,529 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 03:02:41,529 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 canCommit 03:02:41,530 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,530 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 03:02:41,530 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,530 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 03:02:41,530 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] resolved 03:02:41,530 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,530 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,530 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,530 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:41,531 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:41,531 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,531 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 03:02:41,531 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,531 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 03:02:41,531 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] resolved 03:02:41,531 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: successfully resolved all cohort actors 03:02:41,531 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finishCanCommit 03:02:41,531 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,531 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,532 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 03:02:41,532 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,532 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:41,532 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:41,532 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,532 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 03:02:41,532 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:41,532 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit returning result: true 03:02:41,533 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 commit 03:02:41,533 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finish commit 03:02:41,533 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,533 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,533 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:41,533 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,533 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,533 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 03:02:41,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}] 03:02:41,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}] 03:02:41,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:41,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 03:02:41,534 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 282 03:02:41,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 03:02:41,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 03:02:41,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:41,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 03:02:41,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}] 03:02:41,535 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 03:02:41,535 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 03:02:41,536 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 03:02:41,537 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=8321127} is already disabled 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,537 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}]]] 03:02:41,537 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 03:02:41,537 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=7743032} is already disabled 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}]]] 03:02:41,538 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}]]] 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=282} 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:02:41,539 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 03:02:41,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 03:02:41,540 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,542 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$a] 03:02:41,543 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1781773818] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}]]] 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}] 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:02:41,543 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 03:02:41,543 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:02:41,544 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 03:02:41,544 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}] 03:02:41,544 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:41,544 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 03:02:41,544 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 03:02:41,544 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] and replicatedToAllIndex is 0 03:02:41,544 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:02:41,544 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 03:02:41,544 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:02:41,544 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1781773818] 03:02:41,545 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,545 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1781773818] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 03:02:41,545 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 273.1 μs 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,545 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:02:41,545 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%23-581758824$b] 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#377684400] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 03:02:41,546 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7bf73f97, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:02:41,546 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:02:41,546 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 3 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 03:02:41,546 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit succeeded 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}] and replicatedToAllIndex is 0 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=3, lastIndex=1, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 6 03:02:41,546 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:02:41,546 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#377684400] 03:02:41,546 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,546 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#377684400] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 03:02:41,547 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 79.59 μs 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=3, lastAppliedIndex=1, lastAppliedTerm=3, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5812a902, electionTerm=3, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 3 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 6 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 6 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 03:02:41,547 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 03:02:41,547 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 03:02:41,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 03:02:41,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:41,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 03:02:41,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 03:02:41,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 03:02:41,548 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,548 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,548 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,548 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] for shard default, trying to use DirectTransactionContextWrapper 03:02:41,548 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@10834be 03:02:41,548 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 03:02:41,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:41,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 03:02:41,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 03:02:41,548 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 03:02:41,548 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-2103803351] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 03:02:41,549 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:02:41,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:02:41,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 03:02:41,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 03:02:41,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] and replicatedToAllIndex is 0 03:02:41,549 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:02:41,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 03:02:41,549 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:02:41,549 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1877227471] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 03:02:41,549 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:02:41,549 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 3 03:02:41,550 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 03:02:41,550 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=113}] and replicatedToAllIndex is 0 03:02:41,550 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=3, lastIndex=1, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:02:41,550 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 03:02:41,550 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-2103803351] 03:02:41,550 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,550 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1877227471] 03:02:41,551 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,551 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 144.1 μs 03:02:41,551 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 30.26 μs 03:02:41,551 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-2103803351] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:41,551 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1877227471] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:41,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:02:41,551 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@32815c8, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:02:41,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 03:02:41,551 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 03:02:41,551 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:02:41,552 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 03:02:41,552 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 03:02:41,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:02:41,552 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 03:02:41,552 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=3, lastAppliedIndex=1, lastAppliedTerm=3, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@189365eb, electionTerm=3, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:02:41,552 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 03:02:41,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 03:02:41,552 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 3 03:02:41,552 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 03:02:41,552 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 03:02:41,552 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 03:02:41,552 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 03:02:41,552 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 03:02:41,553 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 03:02:41,559 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1864511985] 03:02:41,559 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 03:02:41,559 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] 03:02:41,559 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] 03:02:41,560 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#957403272] 03:02:41,560 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 03:02:41,560 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:41,560 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:41,765 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,765 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,765 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,765 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,765 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying 1 components for commit 03:02:41,765 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying transaction for shard cars 03:02:41,765 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 = [ 1 ] 03:02:41,765 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,766 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,766 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,766 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,766 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 validated 03:02:41,766 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,766 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: success 03:02:41,766 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,766 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,766 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 canCommit 03:02:41,767 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 candidate ready 03:02:41,767 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - candidate rootPath: / 03:02:41,767 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - messages: [] 03:02:41,767 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 03:02:41,767 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 prepared 03:02:41,767 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,767 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,767 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,768 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,768 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 03:02:41,768 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 03:02:41,768 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,768 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:41,768 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,768 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}]]] 03:02:41,768 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 submitted to persistence 03:02:41,769 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 03:02:41,770 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}]]] 03:02:41,770 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,770 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:02:41,770 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,770 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 03:02:41,770 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 03:02:41,770 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,770 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: commit - successfulFromPrevious: [] 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,771 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$c] 03:02:41,772 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 successfully completed direct commit 03:02:41,772 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed successfully 03:02:41,772 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,772 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 03:02:41,772 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,772 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,772 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,772 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:41,772 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 138.0 μs 03:02:41,772 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 03:02:41,772 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,772 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,772 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,772 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,772 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying 1 components for commit 03:02:41,772 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying transaction for shard cars 03:02:41,773 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 = [ 1 ] 03:02:41,773 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,773 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 canCommit 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 validated 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: success 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 candidate ready 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - candidate rootPath: / 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - messages: [] 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: preCommit - successfulFromPrevious: [] 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 prepared 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 03:02:41,773 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,774 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:02:41,774 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,774 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}]]] 03:02:41,774 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 submitted to persistence 03:02:41,774 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 03:02:41,775 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 03:02:41,775 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:02:41,775 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 03:02:41,775 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 03:02:41,775 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 03:02:41,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 03:02:41,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}]]] 03:02:41,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:02:41,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,776 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 03:02:41,777 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,777 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,777 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,777 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 68.65 μs 03:02:41,777 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 03:02:41,779 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 3, nextIndex: 4 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 3 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: commit - successfulFromPrevious: [] 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$d] 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 4 03:02:41,779 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:02:41,780 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 03:02:41,780 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 successfully completed direct commit 03:02:41,780 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed successfully 03:02:41,780 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,780 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 37.50 μs 03:02:41,780 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 03:02:41,780 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,780 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,780 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,780 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,780 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,780 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,780 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,780 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying 1 components for commit 03:02:41,780 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying transaction for shard cars 03:02:41,780 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 = [ 1 ] 03:02:41,780 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,781 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 canCommit 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 validated 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: success 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 candidate ready 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - candidate rootPath: / 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - messages: [] 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: preCommit - successfulFromPrevious: [] 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 prepared 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}]]] 03:02:41,781 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 submitted to persistence 03:02:41,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 3 03:02:41,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 03:02:41,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 3 03:02:41,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 03:02:41,782 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 4 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}]]] 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,783 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,784 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,784 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 67.05 μs 03:02:41,784 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 03:02:41,785 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 03:02:41,785 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 4, nextIndex: 5 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 4 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: commit - successfulFromPrevious: [] 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,785 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$e] 03:02:41,786 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 successfully completed direct commit 03:02:41,786 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed successfully 03:02:41,786 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,786 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 4 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:02:41,786 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,786 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,786 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 03:02:41,786 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 87.66 μs 03:02:41,786 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,786 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,786 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,786 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,786 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying 1 components for commit 03:02:41,786 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying transaction for shard cars 03:02:41,786 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 = [ 1 ] 03:02:41,786 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,786 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 canCommit 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 validated 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: success 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,786 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 candidate ready 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - candidate rootPath: / 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - messages: [] 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: preCommit - successfulFromPrevious: [] 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 prepared 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 03:02:41,787 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 submitted to persistence 03:02:41,788 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 4 03:02:41,788 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 03:02:41,788 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 4 03:02:41,788 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 03:02:41,788 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 4 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,789 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,789 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,789 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 70.73 μs 03:02:41,790 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,790 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:02:41,790 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:02:41,790 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 5, nextIndex: 6 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 5 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: commit - successfulFromPrevious: [] 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,802 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$f] 03:02:41,803 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed successfully 03:02:41,803 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,803 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 successfully completed direct commit 03:02:41,803 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 163.8 μs 03:02:41,803 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 03:02:41,803 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 4 03:02:41,803 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,803 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,803 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,803 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 03:02:41,804 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,804 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,804 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,804 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,804 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying 1 components for commit 03:02:41,804 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying transaction for shard cars 03:02:41,804 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 = [ 1 ] 03:02:41,804 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,804 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 canCommit 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 validated 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: success 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,804 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,805 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 candidate ready 03:02:41,805 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - candidate rootPath: / 03:02:41,805 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - messages: [] 03:02:41,805 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: preCommit - successfulFromPrevious: [] 03:02:41,805 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 prepared 03:02:41,805 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,805 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,805 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,806 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,806 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 03:02:41,806 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 03:02:41,806 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,806 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 03:02:41,806 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,806 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}]]] 03:02:41,806 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 submitted to persistence 03:02:41,807 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 03:02:41,807 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 5 03:02:41,807 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 03:02:41,807 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 5 03:02:41,809 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 03:02:41,809 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 03:02:41,809 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,810 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 03:02:41,811 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 4 03:02:41,811 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}]]] 03:02:41,811 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,811 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 03:02:41,811 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,811 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 03:02:41,811 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,811 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,812 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,812 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 109.4 μs 03:02:41,813 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 03:02:41,813 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 6, nextIndex: 7 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 6 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: commit - successfulFromPrevious: [] 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,813 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$g] 03:02:41,814 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 4 03:02:41,814 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 03:02:41,814 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 03:02:41,814 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed successfully 03:02:41,814 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,814 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 successfully completed direct commit 03:02:41,814 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 03:02:41,814 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,814 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,814 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,815 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,815 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,815 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,815 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,815 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying 1 components for commit 03:02:41,815 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying transaction for shard cars 03:02:41,815 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 = [ 1 ] 03:02:41,815 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,815 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 canCommit 03:02:41,815 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,815 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,816 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 1.505 ms 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 validated 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: success 03:02:41,816 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 6 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,816 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 03:02:41,816 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 6 03:02:41,816 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 03:02:41,816 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 candidate ready 03:02:41,816 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - candidate rootPath: / 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - messages: [] 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: preCommit - successfulFromPrevious: [] 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 prepared 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,816 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 03:02:41,816 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 03:02:41,817 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 54.19 μs 03:02:41,817 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,817 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 03:02:41,817 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 4 03:02:41,817 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,817 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 03:02:41,817 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}]]] 03:02:41,817 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 submitted to persistence 03:02:41,818 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 03:02:41,823 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,823 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}]]] 03:02:41,823 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,823 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 03:02:41,823 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,823 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 03:02:41,824 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,824 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,824 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 7, nextIndex: 8 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 7 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: commit - successfulFromPrevious: [] 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$h] 03:02:41,832 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 4 03:02:41,832 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 03:02:41,832 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 03:02:41,833 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 successfully completed direct commit 03:02:41,833 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed successfully 03:02:41,833 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,833 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 145.6 μs 03:02:41,834 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 03:02:41,834 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,834 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,834 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,835 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,835 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,835 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,835 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,835 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying 1 components for commit 03:02:41,835 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying transaction for shard cars 03:02:41,835 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 = [ 1 ] 03:02:41,835 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,835 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 canCommit 03:02:41,835 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,835 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,835 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,835 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 validated 03:02:41,835 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,835 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: success 03:02:41,835 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,835 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 candidate ready 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - candidate rootPath: / 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - messages: [] 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: preCommit - successfulFromPrevious: [] 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 prepared 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 03:02:41,836 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,837 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: 6 03:02:41,837 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,837 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}]]] 03:02:41,837 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 submitted to persistence 03:02:41,840 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 7 03:02:41,840 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 03:02:41,840 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 7 03:02:41,840 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 03:02:41,840 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 03:02:41,840 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 03:02:41,841 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,841 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 4 03:02:41,841 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}]]] 03:02:41,841 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,841 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 03:02:41,841 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,841 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 03:02:41,841 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,841 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=8, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,841 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 192.6 μs 03:02:41,841 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 03:02:41,844 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 03:02:41,844 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 8, nextIndex: 9 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 7 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 8 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: commit - successfulFromPrevious: [] 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$i] 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 4 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 03:02:41,845 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 03:02:41,847 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 successfully completed direct commit 03:02:41,847 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed successfully 03:02:41,847 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,847 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 120.3 μs 03:02:41,849 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 03:02:41,850 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,850 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,850 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,850 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,850 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,850 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,850 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,850 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying 1 components for commit 03:02:41,850 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying transaction for shard cars 03:02:41,850 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 = [ 1 ] 03:02:41,850 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,850 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 canCommit 03:02:41,850 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,850 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 validated 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: success 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 candidate ready 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - candidate rootPath: / 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - messages: [] 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: preCommit - successfulFromPrevious: [] 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 prepared 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 9, leaderSnapShotIndex: 7 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}]]] 03:02:41,851 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 submitted to persistence 03:02:41,852 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 03:02:41,855 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 8 03:02:41,855 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 7, lastIndex(): 8 03:02:41,855 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 8 03:02:41,855 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 03:02:41,855 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 03:02:41,855 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 03:02:41,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 4 03:02:41,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}]]] 03:02:41,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 03:02:41,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 03:02:41,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,856 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=9, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,856 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 284.8 μs 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 9, nextIndex: 10 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 9, lastAppliedIndex: 8 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 9 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: commit - successfulFromPrevious: [] 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$j] 03:02:41,858 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 successfully completed direct commit 03:02:41,858 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed successfully 03:02:41,858 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,858 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 4 03:02:41,859 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 9, leaderSnapShotIndex: 8 03:02:41,859 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 178.7 μs 03:02:41,859 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 03:02:41,859 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 03:02:41,859 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,859 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,859 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,859 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,859 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,859 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,859 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,860 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying 1 components for commit 03:02:41,860 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying transaction for shard cars 03:02:41,860 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 = [ 1 ] 03:02:41,860 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,860 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 canCommit 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 validated 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: success 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 candidate ready 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - candidate rootPath: / 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - messages: [] 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: preCommit - successfulFromPrevious: [] 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 prepared 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,860 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 03:02:41,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 03:02:41,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 10, leaderSnapShotIndex: 8 03:02:41,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=4, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}]]] 03:02:41,861 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 submitted to persistence 03:02:41,862 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 03:02:41,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 03:02:41,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 03:02:41,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 9 03:02:41,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 9, context.getLastApplied(): 8, lastIndex(): 9 03:02:41,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 9 03:02:41,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 03:02:41,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 03:02:41,864 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 03:02:41,864 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,864 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 106.9 μs 03:02:41,865 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 4 03:02:41,865 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=4, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}]]] 03:02:41,865 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,865 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 9, entries to be added from: 0 03:02:41,865 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,865 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 03:02:41,865 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,865 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,865 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 03:02:41,865 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 10, nextIndex: 11 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 9 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 10 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: commit - successfulFromPrevious: [] 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$k] 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 4 03:02:41,869 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 successfully completed direct commit 03:02:41,869 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 03:02:41,869 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed successfully 03:02:41,869 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,870 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 03:02:41,870 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 132.7 μs 03:02:41,870 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 03:02:41,870 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,870 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,870 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,871 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,871 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,871 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,871 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,871 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying 1 components for commit 03:02:41,871 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying transaction for shard cars 03:02:41,871 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 = [ 1 ] 03:02:41,871 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:41,871 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 canCommit 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 validated 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: success 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 candidate ready 03:02:41,871 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - candidate rootPath: / 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - messages: [] 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: preCommit - successfulFromPrevious: [] 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 prepared 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 11, leaderSnapShotIndex: 9 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}]]] 03:02:41,872 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 submitted to persistence 03:02:41,873 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 10 03:02:41,874 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 9, lastIndex(): 10 03:02:41,874 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 10 03:02:41,874 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 03:02:41,874 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 03:02:41,874 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 03:02:41,874 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 4 03:02:41,874 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 03:02:41,875 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,875 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 85.62 μs 03:02:41,875 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:41,877 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}]]] 03:02:41,877 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:41,877 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 03:02:41,877 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 03:02:41,877 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 03:02:41,877 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:41,877 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=11, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:41,877 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 03:02:41,877 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 11, nextIndex: 12 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 9 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 11 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: commit - successfulFromPrevious: [] 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from COMMIT_PENDING to COMMITTED 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$l] 03:02:41,878 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 successfully completed direct commit 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 4 03:02:41,878 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed successfully 03:02:41,878 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 03:02:41,878 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:41,879 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 119.3 μs 03:02:41,879 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 03:02:41,879 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:41,879 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:41,879 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:41,879 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1864511985] 03:02:41,880 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 03:02:41,880 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] 03:02:41,880 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] 03:02:41,881 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#957403272] 03:02:41,881 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 03:02:41,881 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:41,881 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:41,931 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:41,983 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:42,33 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:42,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 11 03:02:42,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 03:02:42,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 11 03:02:42,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 03:02:42,48 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 03:02:42,49 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 03:02:42,50 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 4 03:02:42,50 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:42,50 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 03:02:42,50 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 206.6 μs 03:02:42,84 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:42,178 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:02:42,287 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:42,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:42,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:42,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] for shard default, trying to use DirectTransactionContextWrapper 03:02:42,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@10834be 03:02:42,287 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:42,288 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying 2 components for commit 03:02:42,288 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard cars 03:02:42,288 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 03:02:42,288 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:42,288 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,288 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard default 03:02:42,288 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 03:02:42,288 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:42,288 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,288 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 canCommit 03:02:42,288 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 03:02:42,288 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,288 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,288 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] resolved 03:02:42,289 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:42,289 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 03:02:42,289 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] resolved 03:02:42,289 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: successfully resolved all cohort actors 03:02:42,289 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finishCanCommit 03:02:42,289 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:42,289 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:42,289 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:42,289 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,289 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,289 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 03:02:42,289 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:42,289 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 03:02:42,290 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:42,290 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:42,290 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:42,290 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,290 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,290 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 03:02:42,290 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:42,290 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 03:02:42,290 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:42,290 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit returning result: true 03:02:42,290 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 commit 03:02:42,290 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finish commit 03:02:42,290 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:42,290 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] 03:02:42,290 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:42,290 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] 03:02:42,291 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,291 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,291 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 03:02:42,291 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 03:02:42,293 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 03:02:42,293 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 03:02:42,293 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 03:02:42,293 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:42,293 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,293 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,294 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 03:02:42,294 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 03:02:42,294 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 03:02:42,294 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,294 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 12, leaderSnapShotIndex: 10 03:02:42,294 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 12 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 03:02:42,294 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}]]] 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 03:02:42,294 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 114 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}] 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}] 03:02:42,294 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:42,295 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:42,295 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 03:02:42,295 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}]]] 03:02:42,295 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 03:02:42,296 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 03:02:42,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}] 03:02:42,299 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}]]] 03:02:42,299 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:42,299 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 11, entries to be added from: 0 03:02:42,299 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 03:02:42,299 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 03:02:42,299 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 03:02:42,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}]]] 03:02:42,299 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:42,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:02:42,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:02:42,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=114} 03:02:42,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}] 03:02:42,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 03:02:42,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:42,299 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}] 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 12, nextIndex: 13 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 12, lastAppliedIndex: 11 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 12 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,300 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%231101536191$m] 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 03:02:42,300 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%23-581758824$n] 03:02:42,300 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:42,301 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:02:42,301 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:42,301 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 4 03:02:42,301 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}] and replicatedToAllIndex is 1 03:02:42,301 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=3, lastIndex=2, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 11 03:02:42,301 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 8 03:02:42,301 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 240.5 μs 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 03:02:42,301 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit succeeded 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] and replicatedToAllIndex is 11 03:02:42,301 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=4, lastIndex=12, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 29 03:02:42,301 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=3, lastAppliedIndex=2, lastAppliedTerm=3, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5cba7d3e, electionTerm=3, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 3 03:02:42,301 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 106.0 μs 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=4, lastAppliedIndex=12, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@aa9e985, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:02:42,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 03:02:42,302 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 4 03:02:42,302 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 8 03:02:42,302 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 8 03:02:42,302 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 03:02:42,302 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 29 03:02:42,302 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 29 03:02:42,302 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 03:02:42,302 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 03:02:42,302 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:42,302 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75819aef 03:02:42,302 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:42,302 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-581758824)] for shard default, trying to use DirectTransactionContextWrapper 03:02:42,302 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@10834be 03:02:42,302 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 03:02:42,302 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:42,302 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 03:02:42,302 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 03:02:42,302 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 03:02:42,304 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1864511985] 03:02:42,304 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 03:02:42,304 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] 03:02:42,304 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191] 03:02:42,304 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#957403272] 03:02:42,304 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 03:02:42,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 12 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 12, context.getLastApplied(): 11, lastIndex(): 12 03:02:42,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 12 03:02:42,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 03:02:42,306 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=114}] and replicatedToAllIndex is 1 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=3, lastIndex=2, lastTerm=3, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:02:42,306 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 03:02:42,307 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:02:42,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 03:02:42,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:02:42,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 4 03:02:42,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 03:02:42,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] and replicatedToAllIndex is 11 03:02:42,307 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=4, lastIndex=12, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:02:42,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 27 03:02:42,307 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:42,307 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:42,307 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:02:42,308 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 28.09 μs 03:02:42,308 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:02:42,308 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 122.3 μs 03:02:42,308 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:02:42,308 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=3, lastAppliedIndex=2, lastAppliedTerm=3, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6d5ddfe4, electionTerm=3, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:02:42,308 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 03:02:42,308 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 3 03:02:42,308 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:02:42,308 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=4, lastAppliedIndex=12, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@610f8fc0, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:02:42,308 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 03:02:42,308 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 4 03:02:42,309 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 03:02:42,309 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 03:02:42,309 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 27 03:02:42,309 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 27 03:02:42,309 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 03:02:42,309 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 03:02:42,309 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-916060594] 03:02:42,310 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 03:02:42,310 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 03:02:42,310 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 03:02:42,310 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 03:02:42,513 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:02:42,514 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:42,514 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 03:02:42,514 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 03:02:42,514 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 03:02:42,514 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 03:02:42,515 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 03:02:42,515 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 03:02:42,515 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c822ed1 03:02:42,515 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c822ed1 03:02:42,515 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 03:02:42,515 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 03:02:42,515 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 03:02:42,515 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ddbd514 03:02:42,515 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@ddbd514 03:02:42,515 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 03:02:42,515 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 03:02:42,515 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 03:02:42,515 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 03:02:42,515 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:42,515 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:42,516 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 03:02:42,518 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite] 03:02:42,518 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 03:02:42,518 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite] 03:02:42,518 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:42,518 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:42,518 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 03:02:42,518 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 03:02:42,519 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 03:02:42,520 PM [cluster-test-shard-dispatcher-16] [WARN] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 209, lastApplied : 12, commitIndex : 12 03:02:42,520 PM [cluster-test-shard-dispatcher-25] [WARN] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 209, lastApplied : 2, commitIndex : 2 03:02:42,520 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 12, lastIndex: 12, isVoting: true 03:02:42,520 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 03:02:42,520 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 03:02:42,520 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 2, lastIndex: 2, isVoting: true 03:02:42,520 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 03:02:42,521 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 03:02:42,521 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 03:02:42,521 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 03:02:42,927 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 03:02:42,927 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] ShardManager - shard-manager-testSnapshotOnRootOverwrite: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#1101536191]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:02:42,927 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 03:02:42,927 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 03:02:42,927 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-167: Closing 03:02:42,927 PM [cluster-test-shard-dispatcher-16] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-168: Closing 03:02:42,928 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 03:02:42,928 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 03:02:42,928 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 03:02:42,928 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 03:02:42,928 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:02:42,928 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 03:02:42,928 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 03:02:42,929 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 03:02:42,929 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-508169139] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-508169139] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:42,929 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:42,929 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:42,941 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:42,942 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:42,942 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:42,942 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:42,944 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:42,958 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 03:02:42,959 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 03:02:42,959 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 03:02:42,959 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 03:02:42,960 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 03:02:42,960 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 03:02:42,961 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 03:02:42,961 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 03:02:42,964 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:42,964 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:43,943 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8977649116481891395] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:43,980 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:43,980 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:43,982 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:43.984] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:43,985 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [115921182919951560] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:44,0 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:44,1 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:02:44.006] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:44,28 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:44,35 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2727266331346496275] 03:02:44,36 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:44,38 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:44,38 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:44,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:44,39 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:44,49 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:44,50 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:44,50 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:44,56 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:44,71 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5607573005686535475] 03:02:44,71 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:44,71 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:44,71 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:44,71 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:44,74 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:44,90 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:44,101 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4975549742796075083] 03:02:44,101 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:44,110 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:44,110 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:44,110 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:44,110 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:44,118 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:44,120 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:44,122 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 03:02:44,140 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 03:02:44,151 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:44,164 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1928837562] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:44,164 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:44,176 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 03:02:44,176 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:02:44,177 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1896b030 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2dbb454a 03:02:44,177 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 03:02:44,177 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2944c914 03:02:44,177 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,177 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:44,177 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,178 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709823764178, None)) 03:02:44,178 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823764177,0,0)) 03:02:44,178 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2dbb454a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@23f0cab0 03:02:44,178 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@30315666 03:02:44,185 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 03:02:44,185 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:44,185 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,186 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:44,187 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:44,187 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:44,189 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 03:02:44,190 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:44,195 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,195 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,195 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309] 03:02:44,195 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 03:02:44,196 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 03:02:44,196 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:44,196 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1161726280] 03:02:44,196 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#2060967774 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,196 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1161726280] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:44,196 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#2060967774] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:44,197 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:44,197 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:44,197 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 03:02:44,197 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 03:02:44,197 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:44,197 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:02:44,197 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 03:02:44,197 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 03:02:44,197 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 03:02:44,204 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#552542588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:44,206 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 03:02:44,206 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:02:44,206 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6af06338 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d0e9cb8 03:02:44,206 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3b4e41b2 03:02:44,207 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 03:02:44,207 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709823764207, None)) 03:02:44,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823764206,0,0)) 03:02:44,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d0e9cb8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1428381c 03:02:44,208 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 03:02:44,208 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:44,209 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:44,209 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,209 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@54c77574 03:02:44,217 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 03:02:44,218 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:44,218 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:44,218 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,218 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:44,218 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1529658873] 03:02:44,219 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 03:02:44,219 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1529658873] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:44,219 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-751445578 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:44,219 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 03:02:44,219 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-751445578] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:44,219 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1123472834] 03:02:44,219 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1123472834] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:44,220 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:44,220 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:44,221 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:44,221 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:02:44,221 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 03:02:44,221 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 03:02:44,221 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,221 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 03:02:44,221 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 03:02:44,221 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 03:02:44,268 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1928837562] 03:02:44,268 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 03:02:44,276 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309] 03:02:44,512 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:44,512 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 03:02:44,512 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 03:02:44,512 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,513 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 03:02:44,513 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:44,513 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 03:02:44,513 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:44,513 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:02:44,513 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 03:02:44,513 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 03:02:44,782 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:02:44,782 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:44,782 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 03:02:44,812 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:45,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:02:45,22 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:45,22 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 03:02:45,53 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:45,53 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:45,55 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:45,55 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 03:02:45,55 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:45,55 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:45,55 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,55 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:45,55 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,68 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:45,68 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 03:02:45,68 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,68 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:45,68 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,68 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,120 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:45,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:02:45,292 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:45,293 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 03:02:45,293 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:45,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 03:02:45,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:45,296 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:45,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 03:02:45,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:45,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:45,296 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:45,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:45,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:45,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 03:02:45,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:45,300 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:45,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:02:45,300 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 03:02:45,300 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 03:02:45,300 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:02:45,301 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:45,301 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:45,301 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 03:02:45,301 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 03:02:45,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 03:02:45,301 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:45,301 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 03:02:45,302 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:02:45,302 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:45,302 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:45,302 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 03:02:45,303 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 03:02:45,303 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:45,335 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithSingleShard commit rateLimit adjusted to 85.79685415538616 03:02:45,335 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,335 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#552542588] 03:02:45,335 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:45,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:45,335 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:45,335 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 03:02:45,335 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:45,335 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:45,335 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 03:02:45,335 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 03:02:45,335 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:45,336 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 03:02:45,336 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 03:02:45,336 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars 03:02:45,336 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 03:02:45,336 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:45,336 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:45,337 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:02:45,337 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 03:02:45,337 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 03:02:45,337 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 4 batched modifications, ready: true 03:02:45,337 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 03:02:45,342 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 03:02:45,342 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 03:02:45,342 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 03:02:45,342 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:45,342 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:45,342 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,342 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 03:02:45,343 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 4 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,343 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 of 4 operations, client version 13 03:02:45,343 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: success 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 245 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 03:02:45,344 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:45,345 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:45,345 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,345 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 03:02:45,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 03:02:45,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:02:45,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:45,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 03:02:45,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 03:02:45,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 03:02:45,345 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:45,346 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 03:02:45,346 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 03:02:45,346 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 03:02:45,346 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:45,346 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,346 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=4948584} is already disabled 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 03:02:45,347 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 03:02:45,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 03:02:45,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:02:45,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:45,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=245} 03:02:45,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 03:02:45,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 03:02:45,348 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:45,349 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:45,349 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$a] 03:02:45,350 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-56643857] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 03:02:45,350 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-56643857] 03:02:45,350 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:02:45,350 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-56643857] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:45,350 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:02:45,350 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 160.0 μs 03:02:45,350 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:45,351 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 03:02:45,351 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 03:02:45,352 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 1 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 03:02:45,352 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 03:02:45,352 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:45,352 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:45,352 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 03:02:45,352 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 03:02:45,352 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 03:02:45,352 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,353 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1244678825] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 03:02:45,353 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1244678825] 03:02:45,353 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:02:45,353 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:02:45,353 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 03:02:45,353 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#1244678825] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:45,353 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 153.1 μs 03:02:45,353 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 03:02:45,354 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-1804541261] 03:02:45,354 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-1804541261] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:45,354 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-1804541261, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0, version=13] 03:02:45,354 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:45,354 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-1804541261)] 03:02:45,356 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 03:02:45,356 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:45,356 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1928837562] 03:02:45,356 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:45,356 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:02:45,357 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1816904c] 03:02:45,357 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@1816904c] 03:02:45,357 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309)] for shard cars 03:02:45,357 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:02:45,357 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@45235d3a 03:02:45,357 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,357 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:45,357 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:45,357 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 03:02:45,357 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:02:45,357 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying 1 components for commit 03:02:45,357 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying transaction for shard cars 03:02:45,357 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 directCommit called 03:02:45,357 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 sending 1 batched modifications, ready: true 03:02:45,357 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 03:02:45,357 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 canCommit 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 of 1 operations, client version 13 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 ready 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 validated 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: success 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 candidate ready 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - candidate rootPath: / 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - messages: [] 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 prepared 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 212 03:02:45,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 03:02:45,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 03:02:45,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:45,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:45,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 03:02:45,359 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 submitted to persistence 03:02:45,361 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 03:02:45,363 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 03:02:45,363 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:02:45,363 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:02:45,363 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=212} 03:02:45,363 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 03:02:45,363 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 03:02:45,363 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:45,364 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 03:02:45,364 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: commit - successfulFromPrevious: [] 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$d] 03:02:45,365 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:45,365 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 162.5 μs 03:02:45,365 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 03:02:45,367 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 successfully completed direct commit 03:02:45,367 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 modifications completed with CommitTransactionReply [version=13] 03:02:45,367 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 03:02:45,367 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:45,367 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:45,367 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 03:02:45,367 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 03:02:45,367 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Queuing TransactionOperation 03:02:45,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 03:02:45,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:02:45,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 03:02:45,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 03:02:45,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 03:02:45,368 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:45,368 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 03:02:45,369 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:02:45,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 03:02:45,369 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 03:02:45,369 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-479224311] 03:02:45,369 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 248.4 μs 03:02:45,369 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-479224311] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:45,370 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-479224311, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0, version=13] 03:02:45,371 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 03:02:45,371 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-479224311)] 03:02:45,373 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 DataExists operation succeeded 03:02:45,373 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 03:02:45,373 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:45,373 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:45,373 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 03:02:45,373 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 03:02:45,373 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Queuing TransactionOperation 03:02:45,374 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 03:02:45,375 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1290477785] 03:02:45,375 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1290477785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:45,375 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1290477785, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0, version=13] 03:02:45,376 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:45,376 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1290477785)] 03:02:45,378 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 ReadData operation succeeded 03:02:45,378 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 03:02:45,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:45,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@45235d3a 03:02:45,378 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 03:02:45,379 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#1928837562] 03:02:45,379 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 03:02:45,379 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309] 03:02:45,379 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1982136309] 03:02:45,430 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:02:45,431 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:45,431 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-173: Closing 03:02:45,431 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:02:45,432 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 03:02:45,433 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:45,433 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:45,442 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:45,442 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:45,443 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:45,443 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:45,447 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:45,464 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:02:45,465 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:45,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 03:02:45,465 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#754558496] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#754558496] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:45,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:02:45,465 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 03:02:45,467 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:45,468 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:46,450 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2727266331346496275] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:46,485 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:46,485 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:46,488 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:46.490] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:46,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5607573005686535475] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:46,516 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:02:46,525 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [-463569935326781275] 03:02:46,525 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:46,528 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:46,528 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 03:02:46,528 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:46,528 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:46,539 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:46,541 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:46,543 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 03:02:46,547 PM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#668100256] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,555 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 03:02:46,558 PM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:02:46,559 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 03:02:46,559 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@43a13bab to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@781a80fe 03:02:46,559 PM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 03:02:46,559 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3aedd68e 03:02:46,559 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:46,560 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709823766559, None)) 03:02:46,560 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 03:02:46,560 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823766558,0,0)) 03:02:46,560 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2a257dbf 03:02:46,560 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@781a80fe to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6b03e1eb 03:02:46,560 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 03:02:46,561 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:46,561 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:46,561 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 03:02:46,562 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 03:02:46,562 PM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1832207148] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,562 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 03:02:46,562 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 03:02:46,563 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#1667432096] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,563 PM [reinstated-member2-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-845136744] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,563 PM [reinstated-member2-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1943568595 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 03:02:46,563 PM [reinstated-member2-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1943568595] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 62 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 245 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 1 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 03:02:46,565 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 03:02:46,566 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1a5c9122}, purged=MutableUnsignedLongSet{size=0}} 03:02:46,566 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:02:46,566 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 1 03:02:46,566 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 03:02:46,566 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 212 03:02:46,566 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 03:02:46,566 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 2 to 2 03:02:46,566 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 03:02:46,567 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:02:46,567 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 03:02:46,567 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 03:02:46,567 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 2.222 ms - Switching actor to Follower - last log index = 2, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 3 03:02:46,567 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 03:02:46,567 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 03:02:46,567 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 03:02:46,568 PM [reinstated-member2-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1911804943] 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:46,568 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:02:46,568 PM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1911804943] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,569 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:02:46,569 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 03:02:46,569 PM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:02:46,569 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:02:46,569 PM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 03:02:46,570 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-2130655895] 03:02:46,571 PM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-2130655895] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 03:02:46,571 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 03:02:46,571 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 03:02:46,571 PM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:02:46,571 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 03:02:46,571 PM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 03:02:46,571 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-845136744 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:46,571 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 03:02:46,571 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 03:02:46,571 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 03:02:46,571 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 03:02:46,572 PM [reinstated-member2-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:02:46,572 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:02:46,572 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:46,572 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:46,572 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 03:02:46,572 PM [reinstated-member2-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 03:02:46,572 PM [reinstated-member2-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 03:02:46,572 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:02:46,572 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 03:02:46,573 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 03:02:46,573 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 03:02:46,573 PM [reinstated-member2-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:02:46,573 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:46,573 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:46,573 PM [reinstated-member2-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 03:02:46,573 PM [reinstated-member2-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 03:02:46,611 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#668100256] 03:02:46,611 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 03:02:46,611 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1832207148] 03:02:46,611 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 03:02:46,612 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#668100256] 03:02:46,612 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 Queuing TransactionOperation 03:02:46,612 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:46,612 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1832207148, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5f6428d] 03:02:46,612 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1832207148, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5f6428d] 03:02:46,612 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0: Found primary ActorSelection[Anchor(akka://reinstated-member2/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1832207148)] for shard cars 03:02:46,612 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:02:46,612 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@453fa600 03:02:46,613 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 03:02:46,613 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:46,613 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 03:02:46,613 PM [reinstated-member2-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 03:02:46,613 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 03:02:46,613 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 03:02:46,614 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@20db3318 03:02:46,614 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@24f48caa 03:02:46,614 PM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@20db3318 03:02:46,614 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@24f48caa 03:02:46,614 PM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 03:02:46,614 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-178: Closing 03:02:46,614 PM [reinstated-member2-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:02:46,614 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-177: Closing 03:02:46,614 PM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:02:46,614 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:02:46,615 PM [reinstated-member2-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 03:02:46,615 PM [reinstated-member2-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 03:02:46,615 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:02:46,615 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:02:46,616 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:02:46,616 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:02:46,616 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 03:02:46,616 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:02:46,616 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:02:46,616 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:02:46,617 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:02:46,616 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#668100256] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#668100256] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,618 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#668100256] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#668100256] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,618 PM [reinstated-member2-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://reinstated-member2/user/$a#1730349705] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#1730349705] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:46,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:46,641 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:46,641 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:46,644 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:46.646] [reinstated-member2-akka.actor.internal-dispatcher-3] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:46,661 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:02:46,668 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7911508387341633231] 03:02:46,668 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:46,670 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:46,670 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:46,677 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:46,677 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:46,682 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:46,682 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:46,682 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:46,687 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:46,694 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3972783288547710913] 03:02:46,695 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:46,696 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:46,697 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:46,698 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:46,698 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:46,708 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 closeTransaction called 03:02:46,708 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_11#-479224311)] 03:02:46,708 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 03:02:46,708 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_10#-1804541261)] 03:02:46,708 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 closeTransaction called 03:02:46,708 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_12#-1290477785)] 03:02:46,734 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:46,747 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:46,753 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:46,762 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8927793334068066804] 03:02:46,765 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:46,772 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:46,772 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:46,772 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:46,772 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:46,775 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:46,777 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:46,779 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,787 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,789 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1897627642] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,805 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 03:02:46,806 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:02:46,806 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@46a8404f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3aac6fe 03:02:46,806 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4eee1c67 03:02:46,806 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1709823766806, None)) 03:02:46,806 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,806 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823766805,0,0)) 03:02:46,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,807 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3aac6fe to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5a082aec 03:02:46,807 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 03:02:46,807 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:46,807 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,807 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@48af09f7 03:02:46,808 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:46,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:46,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,809 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:46,810 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,810 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,810 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#366383606] 03:02:46,810 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 03:02:46,810 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#366383606] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,811 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,813 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 03:02:46,813 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#1255794391] 03:02:46,814 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#1335263808] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,814 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#1255794391] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,816 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:46,816 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:46,816 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#1335263808 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,816 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 03:02:46,816 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 03:02:46,816 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:46,817 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:02:46,817 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 03:02:46,817 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,820 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 03:02:46,822 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,838 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:46,838 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#952132961] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,840 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 03:02:46,840 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:02:46,840 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4c663aaa to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4dd2ddec 03:02:46,842 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:46,850 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1b89223 03:02:46,853 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1709823766852, None)) 03:02:46,853 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,853 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,854 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:46,854 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,854 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823766851,0,0)) 03:02:46,854 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@71360799 03:02:46,854 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4dd2ddec to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@37752626 03:02:46,855 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 03:02:46,855 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:46,855 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,856 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,856 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-486089399] 03:02:46,856 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 03:02:46,857 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 03:02:46,857 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-486089399] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,857 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#651327648 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,858 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#651327648] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,858 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#-686469741] 03:02:46,858 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#-686469741] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:46,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:46,859 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:46,859 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 03:02:46,859 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 03:02:46,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:46,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:02:46,860 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 03:02:46,860 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,860 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:46,860 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 03:02:46,906 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1897627642] 03:02:46,906 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 03:02:46,906 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#366383606] 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:47,118 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:02:47,118 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 03:02:47,119 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 03:02:47,398 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:02:47,398 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,398 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 03:02:47,422 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:47,690 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:47,690 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:47,690 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:47,690 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,690 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,690 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:47,690 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,690 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,690 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,699 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:47,699 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,699 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,699 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:47,699 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,699 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,718 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:02:47,718 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,718 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 03:02:47,718 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,722 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:47,722 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 03:02:47,722 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:47,722 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:47,722 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=null 03:02:47,723 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:47,723 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,723 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:02:47,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:02:47,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:02:47,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 03:02:47,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:47,730 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:02:47,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:02:47,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 03:02:47,731 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 03:02:47,731 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 03:02:47,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:47,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:47,731 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 03:02:47,731 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 03:02:47,732 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:02:47,732 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,732 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 03:02:47,733 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 03:02:47,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:47,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:47,733 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 03:02:47,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 03:02:47,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:47,763 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 03:02:47,763 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#952132961] 03:02:47,763 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Queuing TransactionOperation 03:02:47,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:47,765 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 03:02:47,765 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 03:02:47,765 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars 03:02:47,765 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 03:02:47,765 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 03:02:47,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, disabling transaction tracking 03:02:47,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 03:02:47,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 03:02:47,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:47,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:47,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 03:02:47,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Creating transaction : member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 03:02:47,771 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#2039239004] 03:02:47,771 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#2039239004] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:47,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 03:02:47,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 03:02:47,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Number of entries to be appended = 1 03:02:47,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:47,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 03:02:47,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 03:02:47,772 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#2039239004, transactionId=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0, version=13] 03:02:47,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Log size is now 1 03:02:47,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:47,772 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:47,772 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#2039239004)] 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done true 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: 0, nextIndex: 1 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Setting last applied to 0 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Moving last applied index from -1 to 0 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} does not match any client, pre-disabling client. 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=3131418} is already disabled 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:47,773 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persisting ApplyJournalEntries with index=0 03:02:47,774 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Commit index set to 0 03:02:47,774 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:02:47,774 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Setting last applied to 0 03:02:47,774 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Moving last applied index from -1 to 0 03:02:47,774 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 03:02:47,774 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:02:47,774 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} does not match any client, pre-disabling client. 03:02:47,774 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persisting ApplyJournalEntries with index=0 03:02:47,776 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 ReadData operation succeeded 03:02:47,777 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:02:47,778 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,778 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-183: Closing 03:02:47,778 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 03:02:47,778 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,778 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:47,778 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:47,782 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:47,782 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:47,783 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:47,784 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:47,787 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:47,812 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 03:02:47,913 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,913 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,914 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithCreateTxFailureDueToNoLeader commit rateLimit adjusted to 85.79685415538616 03:02:47,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:02:47,914 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 03:02:47,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:47,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 03:02:47,914 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:47,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 03:02:47,914 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:47,914 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:02:47,914 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 03:02:47,914 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 03:02:47,914 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 03:02:47,915 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying 1 components for commit 03:02:47,915 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying transaction for shard cars 03:02:47,915 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 03:02:47,915 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 canCommit 03:02:47,942 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:02:47,972 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:02:47,973 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 03:02:47,974 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 03:02:48,2 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:02:48,33 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:02:48,33 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 03:02:48,33 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 03:02:48,62 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:02:48,92 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:02:48,92 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 03:02:48,93 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 03:02:48,122 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:02:48,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 03:02:48,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 03:02:48,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 03:02:48,132 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 4 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:02:48,133 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:48,133 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 03:02:48,152 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:02:48,152 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#952132961] 03:02:48,152 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:48,153 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 03:02:48,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:02:48,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:48,333 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 03:02:48,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:48,372 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 03:02:48,372 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 03:02:48,372 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Find primary for shard cars failed org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:02:48,373 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Creating NoOpTransaction because of error org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:02:48,374 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:48,374 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:02:48,377 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:02:48,377 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:02:48,378 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:02:48,378 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:02:48,378 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:02:48,378 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:02:48,379 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:02:48,379 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:02:48,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 03:02:48,379 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:02:48,380 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:48,380 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:48,796 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7911508387341633231] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:49,395 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:49,395 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:49,396 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3972783288547710913] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:49,397 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:49.399] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:49,405 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:49,405 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:49,406 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:49.408] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:49,425 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:49,440 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2471987542400944122] 03:02:49,441 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:49,445 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:49,445 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:49,453 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:49,453 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:49,460 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:49,460 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:49,460 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:49,465 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:49,472 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5470105939460430560] 03:02:49,473 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:49,475 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:49,475 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:49,477 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:49,477 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:49,504 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:02:49,514 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4031653721801739198] 03:02:49,514 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:49,516 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:49,516 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:49,518 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:49,518 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:49,527 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:49,529 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:49,532 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 03:02:49,541 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 03:02:49,545 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-800232975] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,556 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 03:02:49,556 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:02:49,556 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4b38c631 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f31e864 03:02:49,556 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@778a9b66 03:02:49,556 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 03:02:49,556 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,556 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1709823769556, None)) 03:02:49,556 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,556 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823769555,0,0)) 03:02:49,556 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:49,556 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,556 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,557 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7c73c9e 03:02:49,557 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f31e864 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6b801a9b 03:02:49,557 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 03:02:49,557 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:49,557 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,557 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:49,559 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:49,561 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 03:02:49,562 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,562 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,562 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1432251369] 03:02:49,562 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,562 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,562 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,562 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#515544790] 03:02:49,563 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 03:02:49,563 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1432251369] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,563 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1843588247 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,563 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1843588247] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,563 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#717543535] 03:02:49,563 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 03:02:49,564 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#717543535] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,564 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 03:02:49,564 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#905502283 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,572 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#515544790] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,572 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#905502283] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,575 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:49,575 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:49,576 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 03:02:49,576 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 03:02:49,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:49,576 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:02:49,576 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 03:02:49,576 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 03:02:49,576 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 03:02:49,577 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 03:02:49,578 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#1377559602] 03:02:49,578 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#1377559602] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,579 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:49,579 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:49,579 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 03:02:49,579 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 03:02:49,579 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:49,579 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:02:49,579 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 03:02:49,579 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 03:02:49,579 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 03:02:49,600 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:49,601 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:49,606 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:49,614 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 03:02:49,618 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1468710253] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,622 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:49,623 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 03:02:49,623 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:02:49,623 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3794dc1e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72965488 03:02:49,623 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2a994a73 03:02:49,624 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1709823769623, None)) 03:02:49,624 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823769622,0,0)) 03:02:49,625 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72965488 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@524b8712 03:02:49,625 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 03:02:49,641 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 03:02:49,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,641 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,641 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:49,642 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,642 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1b099f44 03:02:49,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:49,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,642 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,643 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1197462088] 03:02:49,643 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,643 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,643 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,643 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-1360943377] 03:02:49,643 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 03:02:49,643 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1197462088] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,643 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 03:02:49,643 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-1360943377] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,644 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-521024229 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,644 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-521024229] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,644 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#841637971] 03:02:49,644 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-218209231 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,644 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-1050550915] 03:02:49,644 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 03:02:49,644 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:49,647 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:49,647 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:02:49,647 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:49,647 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:02:49,647 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 03:02:49,647 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 03:02:49,647 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 03:02:49,647 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 03:02:49,647 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 03:02:49,647 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 03:02:49,649 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 03:02:49,649 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 03:02:49,649 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#841637971] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,649 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-218209231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,649 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#-1050550915] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:49,676 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-800232975] 03:02:49,676 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 03:02:49,684 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1432251369] 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:49,871 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:02:49,872 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 03:02:49,872 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:02:49,890 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 03:02:49,890 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 03:02:50,120 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:02:50,120 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,121 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 03:02:50,141 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:02:50,141 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,141 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 03:02:50,181 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:50,260 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:50,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:02:50,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,430 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 03:02:50,440 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:02:50,440 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,440 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 03:02:50,472 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:50,472 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:50,473 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:50,473 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,473 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,474 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,474 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:50,474 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,474 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,474 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,474 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,474 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,498 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:02:50,670 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,670 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 03:02:50,671 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:50,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 03:02:50,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:50,674 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:50,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 03:02:50,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:50,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:50,676 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:50,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:50,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 03:02:50,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:50,677 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:50,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:02:50,677 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 03:02:50,677 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 03:02:50,677 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 03:02:50,678 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:50,678 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:50,678 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 03:02:50,678 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 03:02:50,678 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,678 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 03:02:50,678 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 03:02:50,678 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:50,678 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:50,679 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:02:50,679 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:50,710 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:02:50,710 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,710 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 03:02:50,710 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:50,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 03:02:50,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:50,712 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:50,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 03:02:50,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:50,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,712 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:50,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:50,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:50,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 03:02:50,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:50,714 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:50,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:02:50,714 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 03:02:50,714 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 03:02:50,715 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 03:02:50,715 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:50,715 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:50,715 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 03:02:50,715 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 03:02:50,716 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 03:02:50,716 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,716 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 03:02:50,716 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:50,716 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:50,716 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 03:02:50,716 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 03:02:50,716 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:02:50,716 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:50,740 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-800232975] 03:02:50,740 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 03:02:50,740 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#515544790] 03:02:50,826 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 03:02:50,836 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 03:02:50,836 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 433283906, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@eaa0e10} 03:02:50,842 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithMultipleShards commit rateLimit adjusted to 85.79685415538616 03:02:50,842 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:50,842 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1468710253] 03:02:50,842 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:50,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:02:50,842 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#1468710253] 03:02:50,842 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 03:02:50,842 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:50,843 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:50,843 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 03:02:50,843 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 03:02:50,843 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 03:02:50,843 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:50,843 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 03:02:50,843 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] for shard people 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:02:50,843 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] for shard cars 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 03:02:50,843 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 03:02:50,844 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 03:02:50,844 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] 03:02:50,844 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 03:02:50,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 03:02:50,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,845 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3997dc9f 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Readying Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 of 1 operations, client version 13 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 ready 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:50,846 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 03:02:50,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:02:50,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 03:02:50,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@26e3f306 03:02:50,847 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 03:02:50,847 PM [cluster-test-shard-dispatcher-12] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:02:50,847 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,848 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,848 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,853 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 03:02:50,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#515544790)] resolved 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 03:02:50,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 03:02:50,864 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:50,868 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a cohort Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:02:50,868 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,868 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 03:02:50,868 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=23004213} is already disabled 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,868 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=22395928} is already disabled 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 03:02:50,869 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:02:50,871 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:02:50,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:02:50,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 03:02:50,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:50,873 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:02:50,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:02:50,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:50,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 03:02:50,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 03:02:50,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 03:02:50,873 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 03:02:50,874 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:02:50,874 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:02:50,874 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@6a013335 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$4.onFailure(ThreePhaseCommitCohortProxy.java:140) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:102) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:95) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:02:50,875 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 abort 03:02:50,876 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a abort cohort path Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:02:50,877 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 03:02:50,874 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:50,874 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 03:02:50,880 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:50,880 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 03:02:50,880 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 03:02:50,880 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 03:02:50,880 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:02:50,880 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:02:50,880 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:02:50,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 03:02:50,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:50,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 03:02:50,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 03:02:50,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 03:02:50,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:02:50,881 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:02:50,885 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@6a013335 is no longer running 03:02:50,886 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:02:50,886 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:50,886 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:50,886 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 03:02:50,887 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 03:02:50,887 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 03:02:50,887 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@57999bfd 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@550d4ff 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@550d4ff 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:50,887 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:50,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards] 03:02:50,889 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:02:50,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:50,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:50,889 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards] 03:02:50,889 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:02:50,889 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:50,889 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:50,891 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:02:50,891 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:02:50,891 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:50,891 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 03:02:51,101 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 03:02:51,101 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:51,104 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testChainedTransactionFailureWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:02:51,104 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:02:51,104 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:51,104 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 03:02:51,300 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 03:02:51,300 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#1432251369]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:02:51,301 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:51,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-191: Closing 03:02:51,301 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:51,301 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-190: Closing 03:02:51,301 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 1 pending queued transactions 03:02:51,301 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 03:02:51,302 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 03:02:51,302 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 03:02:51,302 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:02:51,303 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 03:02:51,303 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 03:02:51,306 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:51,306 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:51,321 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:51,321 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:51,321 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:51,321 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:51,328 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:51,349 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:02:51,349 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 03:02:51,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 03:02:51,349 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 03:02:51,350 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:02:51,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 03:02:51,350 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 03:02:51,350 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 03:02:51,357 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:51,357 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:51,550 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:52,329 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2471987542400944122] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:52,373 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:52,373 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:52,374 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:52.376] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:52,379 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5470105939460430560] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:52,420 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:52,420 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:02:52.424] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:52,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:52,472 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2614783972509512284] 03:02:52,472 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:52,474 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:52,474 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:52,479 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:52,479 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:52,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:52,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:52,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:52,496 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:52,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8976193660984652732] 03:02:52,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:52,516 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:52,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:52,521 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:52,521 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:52,544 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:52,567 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5639903949981415251] 03:02:52,567 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:52,567 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:52,567 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:52,586 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:52,587 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 closeTransaction called 03:02:52,587 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_13#2039239004)] 03:02:52,587 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:52,588 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:52,588 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:52,590 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 03:02:52,600 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-345676008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,601 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 03:02:52,608 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 03:02:52,608 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,608 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,610 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 03:02:52,610 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:52,610 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@12dd47bc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1c1aab97 03:02:52,615 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:52,615 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@18252ccc 03:02:52,621 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:52,621 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709823772621, None)) 03:02:52,621 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823772620,0,0)) 03:02:52,621 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1c1aab97 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@579a6733 03:02:52,622 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 03:02:52,622 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:52,624 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:52,626 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 03:02:52,631 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:52,631 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,631 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,631 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5326428 03:02:52,631 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:52,631 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,637 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1317864522] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,637 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 03:02:52,639 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:52,641 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:52,642 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,642 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,642 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:52,642 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,642 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,642 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,642 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709] 03:02:52,643 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 03:02:52,645 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 03:02:52,645 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 03:02:52,645 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 03:02:52,645 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#1344653992] 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#637018345 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1873291219 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#501771942] 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#637018345] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#1344653992] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1873291219] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,646 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#501771942] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,648 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:52,648 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:52,657 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 03:02:52,657 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 03:02:52,659 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:52,659 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:02:52,660 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:52,660 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:52,660 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:52,660 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:02:52,660 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 03:02:52,660 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 03:02:52,660 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 03:02:52,660 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:02:52,660 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 03:02:52,660 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:02:52,661 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 03:02:52,661 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:52,661 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@680dd168 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2411a482 03:02:52,661 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 03:02:52,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:52,661 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 03:02:52,662 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:52,662 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,662 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,664 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7e2745d3 03:02:52,664 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709823772662, None)) 03:02:52,664 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823772661,0,0)) 03:02:52,664 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6ee4610d 03:02:52,664 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2411a482 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@13e53ffa 03:02:52,664 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 03:02:52,665 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:52,665 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:52,667 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 03:02:52,667 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 03:02:52,674 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:52,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-2-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-693174109] 03:02:52,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 03:02:52,675 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:52,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-2-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#1607674460] 03:02:52,675 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 03:02:52,676 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 03:02:52,676 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#671183885 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 03:02:52,676 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-1896669864] 03:02:52,676 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-214451621 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:52,676 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-2052309158] 03:02:52,676 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 03:02:52,677 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 03:02:52,677 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#1607674460] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,677 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-693174109] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,677 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#671183885] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,677 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-1896669864] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,677 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-214451621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,677 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-2052309158] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:52,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:52,678 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:52,679 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 03:02:52,679 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 03:02:52,679 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:52,679 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:52,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:52,680 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:52,680 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:02:52,680 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,680 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 03:02:52,680 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 03:02:52,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:02:52,681 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,690 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 03:02:52,690 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:02:52,690 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 03:02:52,690 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:02:52,691 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 03:02:52,691 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 03:02:52,715 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-345676008] 03:02:52,716 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 03:02:52,716 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:52,946 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:02:52,946 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:02:52,947 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:52,976 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:02:52,977 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:02:52,977 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 03:02:53,216 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:02:53,216 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,216 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 03:02:53,255 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:02:53,255 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,256 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 03:02:53,486 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:02:53,487 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,487 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 03:02:53,489 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:53,489 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,489 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,498 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:02:53,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,516 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 03:02:53,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,516 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,517 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:53,517 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 03:02:53,517 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:53,517 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:53,517 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 03:02:53,518 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:53,518 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,518 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:02:53,518 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,519 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:02:53,519 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:02:53,519 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 03:02:53,519 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:53,519 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:53,519 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:02:53,520 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:02:53,520 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 03:02:53,520 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:02:53,521 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:53,521 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:53,521 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:53,521 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 03:02:53,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 03:02:53,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,522 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 03:02:53,523 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,523 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:53,523 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:53,523 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:53,524 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:02:53,524 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:53,600 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:53,639 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,640 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 03:02:53,749 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 03:02:53,749 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:53,750 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:02:53,751 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:02:53,751 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 03:02:53,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 03:02:53,753 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-693174109] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 03:02:53,753 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 03:02:53,754 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 03:02:53,754 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 03:02:53,754 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 03:02:53,754 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 03:02:53,754 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:53,754 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:02:53,754 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 03:02:53,754 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Follower 03:02:53,758 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:02:53,806 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,806 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 03:02:53,807 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,807 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,809 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,809 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:53,809 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 03:02:53,809 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:53,809 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 03:02:53,809 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 03:02:53,809 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:53,809 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,810 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:53,813 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:53,813 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:53,813 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 03:02:53,813 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:53,813 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:53,813 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:53,813 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:02:53,813 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 03:02:53,814 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:02:53,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 03:02:53,815 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,815 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:53,815 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,815 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:53,815 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:53,815 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:02:53,815 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:53,815 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:02:53,815 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:53,815 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:53,815 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 03:02:53,815 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:02:53,817 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:02:53,817 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:53,874 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-345676008] 03:02:53,874 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 03:02:53,874 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709] 03:02:53,874 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:53,878 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:53,882 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 03:02:53,884 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,889 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1400691673] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,890 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 03:02:53,890 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 03:02:53,890 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,891 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,891 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 03:02:53,891 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:53,891 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@50f0623f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@747d975d 03:02:53,891 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@591cb08a 03:02:53,891 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709823773891, None)) 03:02:53,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823773890,0,0)) 03:02:53,892 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@747d975d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3b4325b1 03:02:53,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1b7f7961 03:02:53,892 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 03:02:53,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:53,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,893 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testLeadershipTransferOnShutdown commit rateLimit adjusted to 85.79685415538616 03:02:53,893 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,893 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1317864522] 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#643866008] 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#643866008] 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#643866008] 03:02:53,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,893 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 03:02:53,893 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 03:02:53,893 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1317864522] 03:02:53,893 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 03:02:53,893 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying 2 components for commit 03:02:53,893 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard cars 03:02:53,893 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 03:02:53,893 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard people 03:02:53,893 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 03:02:53,894 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 03:02:53,894 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-345676008] 03:02:53,894 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 03:02:53,894 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:53,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:53,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 03:02:53,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:02:53,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 03:02:53,894 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 03:02:53,894 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 03:02:53,894 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 2 ] 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 2 batched modifications, ready: true 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 03:02:53,895 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] for shard people 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 1 ] 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 1 batched modifications, ready: true 03:02:53,895 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] 03:02:53,895 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 03:02:53,895 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:53,896 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 03:02:53,896 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1371609259 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-1856872344] 03:02:53,896 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#643866008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:53,896 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#1140137212 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,896 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-85550390] 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#858262034] 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#858262034] 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#858262034] 03:02:53,902 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:53,902 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:53,902 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 03:02:53,902 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 03:02:53,902 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:53,902 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:02:53,902 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,902 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,902 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 03:02:53,902 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:02:53,903 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 03:02:53,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:53,903 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:53,903 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 03:02:53,903 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 03:02:53,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:53,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:02:53,903 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 03:02:53,903 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,903 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,903 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:02:53,903 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 03:02:53,906 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 03:02:53,906 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,906 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 2 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 2 operations, client version 13 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,907 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 1 operations, client version 13 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:53,908 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 03:02:53,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,910 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 03:02:53,910 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 3 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 03:02:53,910 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 03:02:53,910 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:53,910 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:02:53,910 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:02:53,911 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 03:02:53,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:53,911 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:53,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:02:53,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:53,911 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:02:53,911 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:53,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,911 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,911 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 03:02:53,911 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 03:02:53,911 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:53,911 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:53,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 03:02:53,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:02:53,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 03:02:53,911 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:02:53,913 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,913 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 03:02:53,913 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=7922337} is already disabled 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 03:02:53,915 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,915 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=9113057} is already disabled 03:02:53,916 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,916 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:02:53,916 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:02:53,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 03:02:53,918 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 03:02:53,918 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:02:53,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:02:53,918 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 03:02:53,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 03:02:53,918 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:02:53,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:02:53,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,918 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:53,918 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:53,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 03:02:53,918 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 03:02:53,918 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:02:53,918 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:02:53,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:53,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:02:53,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:53,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:53,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:02:53,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:53,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,918 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 03:02:53,919 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:53,919 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 03:02:53,919 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:53,919 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,919 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:53,920 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 03:02:53,920 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 03:02:53,920 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 03:02:53,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 03:02:53,920 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:53,921 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 03:02:53,922 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:02:53,946 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:53,946 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:53,947 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:53,947 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:53,947 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 03:02:53,947 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:02:53,947 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying 1 components for commit 03:02:53,947 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying transaction for shard cars 03:02:53,947 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 directCommit called 03:02:53,947 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 sending 1 batched modifications, ready: true 03:02:53,947 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 03:02:53,948 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-345676008] 03:02:53,948 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 03:02:53,948 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:53,948 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:53,950 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:53,950 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 of 1 operations, client version 13 03:02:53,950 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:53,950 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 ready 03:02:53,950 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 is scheduled for canCommit step 03:02:53,999 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:54,0 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 03:02:54,0 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-345676008] 03:02:54,0 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:54,1 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 03:02:54,1 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 03:02:54,1 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 03:02:54,1 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 03:02:54,1 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521] 03:02:54,1 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 canCommit 03:02:54,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 03:02:54,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 03:02:54,1 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:02:54,1 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521)] resolved 03:02:54,1 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:54,1 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:54,1 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709)] resolved 03:02:54,1 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: successfully resolved all cohort actors 03:02:54,1 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@317e4ea7 03:02:54,1 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finishCanCommit 03:02:54,1 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521)] 03:02:54,1 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521)] 03:02:54,3 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 03:02:54,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 03:02:54,3 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:02:54,3 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:54,4 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:54,4 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:02:54,4 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:54,4 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:54,4 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,4 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,4 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 03:02:54,4 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:54,4 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 03:02:54,5 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:54,5 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709)] 03:02:54,5 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709)] 03:02:54,6 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,6 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,6 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 03:02:54,6 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:54,6 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 03:02:54,7 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:54,7 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit returning result: true 03:02:54,7 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 commit 03:02:54,7 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finish commit 03:02:54,7 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521)] 03:02:54,7 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-914281521)] 03:02:54,7 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709)] 03:02:54,7 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#713623709)] 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 169 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 validated 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: success 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:54,9 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:54,9 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 03:02:54,10 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}]]] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 candidate ready 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - candidate rootPath: / 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - messages: [] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 prepared 03:02:54,10 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:54,10 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 scheduled for commit step 03:02:54,10 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}]]] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 03:02:54,10 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 228 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 03:02:54,10 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 submitted to persistence 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}]]] 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 03:02:54,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}]]] 03:02:54,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 03:02:54,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:54,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 03:02:54,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 03:02:54,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 03:02:54,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 03:02:54,12 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 03:02:54,12 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 03:02:54,13 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 03:02:54,13 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 03:02:54,13 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 03:02:54,13 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:54,13 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 03:02:54,13 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 03:02:54,13 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 03:02:54,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 03:02:54,14 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 03:02:54,14 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:54,16 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-people-testLeadershipTransferOnShutdown%23713623709$g] 03:02:54,16 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown%23-914281521$f] 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1485000683] with name member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 03:02:54,17 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-660776278] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 03:02:54,17 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 3 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 03:02:54,17 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: commit - successfulFromPrevious: [] 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown$c] 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@317e4ea7 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:54,17 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:54,17 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1485000683] 03:02:54,18 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 4 03:02:54,18 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-660776278] 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1485000683] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-660776278] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 142.5 μs 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 109.5 μs 03:02:54,18 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 44.51 μs 03:02:54,19 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit succeeded 03:02:54,19 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=13] 03:02:54,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 03:02:54,19 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 03:02:54,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 03:02:54,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 03:02:54,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:02:54,19 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 03:02:54,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 03:02:54,19 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 03:02:54,19 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:02:54,19 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 03:02:54,19 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,20 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1535234866] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 03:02:54,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 03:02:54,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 03:02:54,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 03:02:54,20 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 2, lastIndex: 2, isVoting: true 03:02:54,20 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 03:02:54,20 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:54,20 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:54,20 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:54,20 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 03:02:54,21 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 03:02:54,21 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 03:02:54,21 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1535234866] 03:02:54,21 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:02:54,21 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1952578817] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 03:02:54,21 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 115.2 μs 03:02:54,21 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 03:02:54,21 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:02:54,21 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 03:02:54,21 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 03:02:54,21 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 43.97 μs 03:02:54,21 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:02:54,22 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1535234866] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:54,22 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 03:02:54,22 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 03:02:54,22 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 4 03:02:54,22 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1952578817] 03:02:54,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 4 03:02:54,22 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:02:54,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 03:02:54,23 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 57.19 μs 03:02:54,23 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:54,23 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 38.67 μs 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:02:54,23 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 03:02:54,23 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 canCommit 03:02:54,23 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 successfully completed direct commit 03:02:54,23 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:02:54,24 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:02:54,24 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 03:02:54,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:54,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 03:02:54,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 03:02:54,24 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 03:02:54,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 03:02:54,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 03:02:54,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:54,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:02:54,24 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:54,24 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 03:02:54,24 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-199: Closing 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 03:02:54,25 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:02:54,25 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 03:02:54,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 03:02:54,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 03:02:54,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 03:02:54,26 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:54,27 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:54,27 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:02:54,27 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:02:54,27 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:54,27 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 03:02:54,27 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:54,27 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:54,27 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:02:54,27 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 03:02:54,27 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:02:54,27 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 03:02:54,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:54,28 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:54,28 PM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:54,28 PM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:54,28 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 26.51 ms 03:02:54,28 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:02:54,28 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 03:02:54,28 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:02:54,28 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:54,28 PM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:02:54,28 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:02:54,29 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 03:02:54,29 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1317864522] 03:02:54,29 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:54,29 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Queuing TransactionOperation 03:02:54,29 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-693174109, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@77e51066] 03:02:54,29 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-693174109, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@77e51066] 03:02:54,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 03:02:54,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:54,29 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-693174109)] for shard cars 03:02:54,29 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:02:54,29 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4c925090 03:02:54,29 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:02:54,29 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,29 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 03:02:54,29 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1400691673] 03:02:54,29 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:54,29 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 03:02:54,30 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:54,30 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 03:02:54,30 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:54,30 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 03:02:54,30 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 03:02:54,30 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:54,30 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:54,30 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:02:54,30 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 03:02:54,30 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 03:02:54,30 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:54,30 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 03:02:54,30 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown)] found - trying create transaction 03:02:54,30 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown)] 03:02:54,31 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-693174109] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-1-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:54,32 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Creating transaction : member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,33 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-354304086] 03:02:54,33 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-354304086] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:54,33 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 03:02:54,33 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:02:54,33 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:02:54,33 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:54,33 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:54,33 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 2 03:02:54,33 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:54,34 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-354304086, transactionId=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=13] 03:02:54,34 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:54,34 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-354304086)] 03:02:54,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:54,35 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 03:02:54,35 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 3, nextIndex: 4 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 3 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=3608460} is already disabled 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 2 for term 5 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:02:54,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 03:02:54,39 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ReadData operation succeeded 03:02:54,39 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 3 03:02:54,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 3 03:02:54,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 03:02:54,40 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 03:02:54,40 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 2 for term 5 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 03:02:54,40 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 03:02:54,41 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 03:02:54,41 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:02:54,41 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:02:54,42 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 03:02:54,42 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 03:02:54,42 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:02:54,42 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 03:02:54,42 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 03:02:54,43 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:02:54,43 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:54,43 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 03:02:54,43 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:02:54,43 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 03:02:54,43 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 03:02:54,44 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 03:02:54,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 03:02:54,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 03:02:54,44 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:02:54,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:54,44 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:54,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@54d62334 03:02:54,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@54d62334 03:02:54,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 03:02:54,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:54,44 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:54,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown] 03:02:54,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 03:02:54,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:54,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 03:02:54,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:02:54,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 03:02:54,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:02:54,46 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:02:54,46 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:54,46 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:54,46 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1684571270] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 03:02:54,47 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1684571270] 03:02:54,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 3 03:02:54,47 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 03:02:54,47 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:02:54,47 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 104.2 μs 03:02:54,47 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1684571270] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:54,52 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testLeadershipTransferOnShutdown, matchIndex: 1, lastIndex: 1, isVoting: true 03:02:54,52 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 03:02:54,52 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:54,52 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:54,52 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 03:02:54,54 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 03:02:54,54 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 03:02:54,54 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 03:02:54,55 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 03:02:54,55 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=3] to peer member-1-shard-people-testLeadershipTransferOnShutdown 03:02:54,55 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=3] to peer member-3-shard-people-testLeadershipTransferOnShutdown 03:02:54,55 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 4 03:02:54,55 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 03:02:54,55 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:54,55 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:02:54,55 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 03:02:54,55 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:02:54,56 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:02:54,56 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=3] with higher term appears reachable, updating term. 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=3]" message is greater than leader's term 3 - switching to Follower 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=4, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 1, lastTerm: 3 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 03:02:54,57 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:54,58 PM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 4 03:02:54,58 PM [cluster-test-shard-dispatcher-34] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-198: Closing 03:02:54,58 PM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:02:54,58 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:02:54,58 PM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 03:02:54,58 PM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 03:02:54,58 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Leader to Follower 03:02:54,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:54,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:54,58 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 03:02:54,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:54,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:54,59 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:54,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:02:54,59 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 03:02:54,59 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:02:54,60 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:02:54,60 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:54,60 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:54,60 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 03:02:54,60 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:02:54,60 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:02:54,60 PM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Leader to Follower 03:02:54,60 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 03:02:54,60 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:02:54,60 PM [cluster-test-shard-dispatcher-31] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-people-testLeadershipTransferOnShutdown in 16.56 ms 03:02:54,60 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 03:02:54,61 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-people-testLeadershipTransferOnShutdown status sync done false 03:02:54,61 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 03:02:54,61 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 03:02:54,63 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:02:54,63 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 03:02:54,63 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:54,63 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 03:02:54,64 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 03:02:54,64 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:02:54,64 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:02:54,64 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 03:02:54,64 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:02:54,64 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:02:54,65 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:54,65 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:54,69 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:54,70 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:54,70 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:54,70 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:54,73 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:54,92 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:02:54,92 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 03:02:54,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-204: Closing 03:02:54,93 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 03:02:54,93 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-203: Closing 03:02:54,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:02:54,93 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:02:54,94 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 03:02:54,95 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:54,95 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:55,79 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2614783972509512284] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:55,109 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8976193660984652732] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:55,113 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:55,120 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:55,122 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:55.124] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:55,143 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:55,144 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:55,146 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:55.148] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:55,173 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:02:55,183 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1230194702964618941] 03:02:55,183 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:55,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:55,185 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:55,197 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:55,197 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:55,209 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:55,209 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:55,209 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:55,214 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:55,223 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [709163680735081775] 03:02:55,223 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:55,225 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:55,225 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:55,226 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:55,226 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:55,243 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:55,266 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5570808849711154058] 03:02:55,266 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:55,274 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:55,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:55,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:55,274 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:55,278 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:55,280 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:55,281 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 03:02:55,296 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 03:02:55,303 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,314 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 03:02:55,314 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,314 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,314 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 03:02:55,314 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:55,314 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@55230cb8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5f8d143a 03:02:55,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:55,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,316 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,318 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2dedfa16 03:02:55,318 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1709823775314, None)) 03:02:55,319 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823775313,0,0)) 03:02:55,319 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@69783f37 03:02:55,319 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5f8d143a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@39c7ed55 03:02:55,319 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 03:02:55,320 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:55,320 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,320 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:55,322 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:55,323 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:55,323 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,323 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 03:02:55,324 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 03:02:55,324 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 03:02:55,324 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 03:02:55,325 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 03:02:55,326 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,326 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,326 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1825253582] 03:02:55,326 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-485366870 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,327 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-485366870] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,327 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1825253582] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,327 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#361202184] 03:02:55,327 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1289862095 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,327 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1289862095] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,327 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#361202184] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,327 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,328 PM [cluster-test-shard-dispatcher-15] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,328 PM [cluster-test-shard-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026] 03:02:55,328 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:55,329 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 03:02:55,330 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:55,330 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:55,330 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:55,330 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:55,330 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:55,330 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:55,330 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 03:02:55,330 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 03:02:55,330 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:55,330 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:55,330 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 03:02:55,330 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 03:02:55,330 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 03:02:55,332 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 03:02:55,332 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,333 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,335 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 03:02:55,335 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 03:02:55,335 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 03:02:55,335 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 03:02:55,335 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 03:02:55,337 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:55,340 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:55,344 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:55,344 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,344 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,345 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:55,356 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 03:02:55,356 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:55,356 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7c3feafe to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72f80b09 03:02:55,356 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@76c7818e 03:02:55,358 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1709823775358, None)) 03:02:55,358 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823775357,0,0)) 03:02:55,358 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7449735e 03:02:55,358 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72f80b09 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7423a9f4 03:02:55,358 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 03:02:55,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:55,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,360 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:55,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,361 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,361 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1708534420] 03:02:55,361 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 03:02:55,361 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,362 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1708534420] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-2109908185 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1534625880] 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#610150275] 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1391140157 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1391140157] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-2109908185] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1534625880] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,362 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#610150275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:55,363 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 03:02:55,363 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 03:02:55,364 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:55,364 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:55,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 03:02:55,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 03:02:55,364 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:55,364 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:55,364 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:55,364 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:55,364 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 03:02:55,364 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,364 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 03:02:55,364 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 03:02:55,365 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 03:02:55,365 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 03:02:55,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:55,365 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:55,365 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,365 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 03:02:55,365 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 03:02:55,365 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 03:02:55,410 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:55,410 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:55,410 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:55,599 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:55,600 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 03:02:55,600 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 03:02:55,628 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:55,629 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 03:02:55,629 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 03:02:55,629 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,629 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 03:02:55,629 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:55,629 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:02:55,629 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:55,629 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:55,629 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 03:02:55,630 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 03:02:55,858 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:02:55,858 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:55,858 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 03:02:55,888 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:02:55,888 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:55,889 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 03:02:55,903 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:55,913 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:56,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:02:56,138 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,139 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 03:02:56,179 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:02:56,179 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,179 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 03:02:56,209 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:56,209 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,210 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,249 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,458 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:02:56,458 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,458 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 03:02:56,459 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:56,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 03:02:56,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:56,461 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:56,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 03:02:56,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:56,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,461 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:56,464 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:56,464 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:56,464 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 03:02:56,465 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:56,465 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:56,465 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 03:02:56,465 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 03:02:56,465 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 03:02:56,465 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:02:56,465 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:56,465 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:56,465 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 03:02:56,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,466 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,466 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 03:02:56,467 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:02:56,467 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:56,467 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:56,470 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 03:02:56,470 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:56,488 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:02:56,489 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,489 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 03:02:56,489 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:56,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 03:02:56,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:56,490 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:56,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 03:02:56,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:56,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,491 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:56,492 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:56,492 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:56,492 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 03:02:56,492 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:56,493 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:02:56,493 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 03:02:56,493 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 03:02:56,493 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 03:02:56,493 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:02:56,493 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:56,493 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:56,493 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 03:02:56,493 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:02:56,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,494 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 03:02:56,494 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:02:56,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:56,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:56,494 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:02:56,495 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 03:02:56,495 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:56,570 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:56,571 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 03:02:56,571 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026] 03:02:56,572 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:56,572 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,572 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,572 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,573 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,573 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,573 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,573 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,573 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionForwardedToLeaderAfterRetry commit rateLimit adjusted to 85.79685415538616 03:02:56,574 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,574 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,574 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 03:02:56,574 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,574 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 03:02:56,574 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying 2 components for commit 03:02:56,574 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard cars 03:02:56,574 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 03:02:56,574 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard people 03:02:56,574 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 03:02:56,574 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 canCommit 03:02:56,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:56,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:02:56,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:02:56,574 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:02:56,574 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 03:02:56,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:02:56,576 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 03:02:56,576 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 03:02:56,576 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 03:02:56,576 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,580 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 13 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,581 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 13 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,581 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 03:02:56,582 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,583 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,583 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 03:02:56,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 03:02:56,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:56,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 03:02:56,583 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 03:02:56,583 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,583 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,583 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:56,583 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,583 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,584 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 03:02:56,584 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] resolved 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] resolved 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: successfully resolved all cohort actors 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finishCanCommit 03:02:56,584 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,584 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,584 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 03:02:56,584 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=5008495} is already disabled 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=5311922} is already disabled 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:56,586 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,586 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 03:02:56,587 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 03:02:56,588 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:02:56,588 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 03:02:56,588 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 03:02:56,588 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,588 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,588 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 03:02:56,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 03:02:56,588 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 03:02:56,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:02:56,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 03:02:56,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 03:02:56,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 03:02:56,588 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 03:02:56,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:56,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,589 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,589 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,589 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 03:02:56,589 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,589 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 03:02:56,590 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:56,590 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit returning result: true 03:02:56,590 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 commit 03:02:56,590 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finish commit 03:02:56,590 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,590 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,590 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,590 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:02:56,591 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,591 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 03:02:56,592 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 03:02:56,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 03:02:56,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 03:02:56,592 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:02:56,592 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:02:56,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:56,592 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:56,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:56,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 03:02:56,592 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:56,592 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,592 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 03:02:56,592 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 03:02:56,592 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 03:02:56,593 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 03:02:56,593 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:02:56,593 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 03:02:56,593 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,593 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:56,593 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=183} 03:02:56,593 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 03:02:56,593 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 03:02:56,593 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,593 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:56,593 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:02:56,593 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=179} 03:02:56,593 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:02:56,593 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,593 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:02:56,593 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 03:02:56,594 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:02:56,594 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 03:02:56,594 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:56,594 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-106098026$f] 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-1511764914$e] 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-849630739] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1573606815] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:56,595 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-849630739] 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 03:02:56,595 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:56,595 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1573606815] 03:02:56,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 03:02:56,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,595 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-849630739] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:56,595 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1573606815] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:56,596 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 89.11 μs 03:02:56,596 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 120.5 μs 03:02:56,596 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit succeeded 03:02:56,599 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,599 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,599 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,599 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 03:02:56,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:56,600 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 03:02:56,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 03:02:56,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 03:02:56,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 03:02:56,600 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:02:56,600 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 03:02:56,600 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 03:02:56,600 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 03:02:56,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,601 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,601 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#2076932136] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 03:02:56,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 03:02:56,602 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2080000626] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 03:02:56,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 03:02:56,602 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 03:02:56,602 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2080000626] 03:02:56,602 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 03:02:56,602 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,602 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 100.7 μs 03:02:56,602 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#2076932136] 03:02:56,602 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,602 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2080000626] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:56,602 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#2076932136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:56,603 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 277.1 μs 03:02:56,649 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,650 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,650 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 03:02:56,651 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1708534420] 03:02:56,651 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1708534420] 03:02:56,651 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,651 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:56,651 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:56,651 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 03:02:56,651 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:56,651 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 03:02:56,651 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:56,651 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 03:02:56,651 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:02:56,651 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying 2 components for commit 03:02:56,651 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard cars 03:02:56,651 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 03:02:56,651 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 03:02:56,651 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 03:02:56,651 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,652 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard people 03:02:56,652 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 03:02:56,652 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 03:02:56,652 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 03:02:56,652 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,652 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 canCommit 03:02:56,654 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,654 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 03:02:56,654 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,654 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 03:02:56,654 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,654 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 03:02:56,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,655 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=13] 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] resolved 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] resolved 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: successfully resolved all cohort actors 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 finishCanCommit 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,665 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,667 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,667 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,667 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 03:02:56,667 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,667 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 03:02:56,668 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:56,668 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,668 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,669 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,670 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,670 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 03:02:56,670 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,670 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 03:02:56,671 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:56,671 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit returning result: true 03:02:56,671 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,671 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:56,671 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:56,671 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 03:02:56,672 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 03:02:56,672 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 03:02:56,672 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:56,672 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 03:02:56,672 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 03:02:56,672 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying 2 components for commit 03:02:56,672 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard cars 03:02:56,672 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 03:02:56,672 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 03:02:56,672 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 03:02:56,672 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,672 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard people 03:02:56,672 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 03:02:56,672 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 03:02:56,672 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 03:02:56,672 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,672 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:56,673 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,673 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,673 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,673 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,673 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,673 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,673 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,673 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,673 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:56,673 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:56,673 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 03:02:56,674 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}] 03:02:56,674 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}] 03:02:56,674 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 2 batched modifications, ready: false 03:02:56,674 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,674 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}] 03:02:56,674 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car5}] 03:02:56,676 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,676 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 03:02:56,676 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,676 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 03:02:56,676 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,676 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 03:02:56,676 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,676 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 03:02:56,676 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,678 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 03:02:56,678 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=13] 03:02:56,678 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 03:02:56,678 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=13] 03:02:56,678 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 03:02:56,678 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 2 batched modifications, ready: false 03:02:56,678 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,679 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car6}] 03:02:56,679 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,679 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:56,679 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:56,679 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 03:02:56,679 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car7}] 03:02:56,679 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,679 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:56,679 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:56,679 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 03:02:56,679 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,679 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Queuing TransactionOperation 03:02:56,679 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:56,679 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,679 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,680 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,681 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,681 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,682 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#-779559026] 03:02:56,682 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#-779559026] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:56,683 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 03:02:56,683 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#-779559026, transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, version=13] 03:02:56,683 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}] 03:02:56,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,730 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,730 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,731 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 03:02:56,732 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,732 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:56,732 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,733 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-212: Closing 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,733 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-211: Closing 03:02:56,733 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:56,733 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 03:02:56,733 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 03:02:56,733 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 03:02:56,733 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 03:02:56,733 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:56,733 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 03:02:56,782 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:56,782 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,782 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,782 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 canCommit 03:02:56,783 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Readying 1 components for commit 03:02:56,783 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 Readying transaction for shard cars 03:02:56,783 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 directCommit called 03:02:56,783 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 sending 1 batched modifications, ready: true 03:02:56,783 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,783 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Readying 1 components for commit 03:02:56,783 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 Readying transaction for shard cars 03:02:56,783 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 directCommit called 03:02:56,783 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 sending 1 batched modifications, ready: true 03:02:56,783 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,783 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Readying 1 components for commit 03:02:56,783 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 Readying transaction for shard cars 03:02:56,783 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 directCommit called 03:02:56,783 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 sending 1 batched modifications, ready: true 03:02:56,783 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-2:datastore-testTransactionForwardedToLeaderAfterRetry@0:5_15#-779559026)] 03:02:56,784 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 03:02:56,784 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] resolved 03:02:56,784 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] resolved 03:02:56,784 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: successfully resolved all cohort actors 03:02:56,784 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 finishCanCommit 03:02:56,785 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,785 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 03:02:56,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:56,790 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, ready=true, participatingShardNames=null, totalMessagesSent=3, modifications size=1] to retry 03:02:56,791 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,791 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, ready=true, participatingShardNames=null, totalMessagesSent=1, modifications size=1] to retry 03:02:56,791 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,791 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to retry 03:02:56,791 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: handleForwardedReadyTransaction for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,791 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Adding message ForwardedReadyTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, transaction=ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, closed=false, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}], operation=WRITE}}}}}}}]}, doImmediateCommit=true, participatingShardNames=null, txnClientVersion=13] to retry 03:02:56,835 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,835 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,835 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,835 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 03:02:56,835 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 03:02:56,835 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,835 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,836 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:56,836 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 03:02:56,837 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:56,837 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 03:02:56,837 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:02:56,837 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 03:02:56,837 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 03:02:56,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 03:02:56,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,838 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:02:56,839 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 03:02:56,839 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 03:02:56,839 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 03:02:56,839 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:56,840 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:02:56,840 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 03:02:56,840 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 03:02:56,840 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 03:02:56,840 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:02:56,840 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:56,840 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:56,840 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 03:02:56,840 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:02:56,844 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,844 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,844 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:02:56,845 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:56,845 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:56,846 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 pending transaction messages to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,846 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 03:02:56,846 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 03:02:56,846 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 03:02:56,846 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, ready=true, participatingShardNames=null, totalMessagesSent=3, modifications size=1] 03:02:56,846 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, ready=true, participatingShardNames=null, totalMessagesSent=1, modifications size=1] 03:02:56,846 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] 03:02:56,847 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Retrying message ForwardedReadyTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, transaction=ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, closed=false, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car8}], operation=WRITE}}}}}}}]}, doImmediateCommit=true, participatingShardNames=null, txnClientVersion=13] 03:02:56,847 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,847 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 BatchedModifications to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding 1 BatchedModifications to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CanCommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: handleForwardedReadyTransaction for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,848 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 03:02:56,851 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 2 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 of 5 operations, client version 13 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 ready 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 is scheduled for canCommit step 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 of 1 operations, client version 13 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 ready 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 is scheduled for canCommit step 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 is scheduled for canCommit step 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 of 1 operations, client version 13 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,852 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 ready 03:02:56,853 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 is scheduled for canCommit step 03:02:56,855 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 modifications completed with BatchedModificationsReply [numBatched=2] 03:02:56,855 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.datastore.messages.BatchedModificationsReply] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] to Actor[akka://cluster-test/deadLetters] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:56,857 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 03:02:56,885 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 03:02:56,886 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1708534420] 03:02:56,886 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 commit 03:02:56,886 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 finish commit 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 03:02:56,886 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 03:02:56,886 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,886 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 03:02:56,886 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:02:56,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,887 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 03:02:56,887 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 03:02:56,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:56,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:56,887 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 03:02:56,889 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,889 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,889 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:56,889 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 03:02:56,889 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:02:56,889 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 03:02:56,889 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 03:02:56,889 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 03:02:56,889 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,890 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:02:56,891 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Adding message CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] to retry 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 candidate ready 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - candidate rootPath: / 03:02:56,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 03:02:56,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - messages: [] 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 prepared 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 77 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 validated 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: success 03:02:56,892 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 03:02:56,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:56,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 03:02:56,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:56,893 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 03:02:56,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:56,893 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 03:02:56,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,893 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:02:56,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}]]] 03:02:56,893 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:56,893 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:56,893 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 submitted to persistence 03:02:56,893 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 03:02:56,893 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:02:56,893 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding 3 pending transaction messages to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding pending transaction message BatchedModifications [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, ready=true, participatingShardNames=[cars, people], totalMessagesSent=1, modifications size=1] 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Retrying message CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=13] 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,898 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CanCommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}]]] 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=77} 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:02:56,899 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,900 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=77}] 03:02:56,900 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:02:56,900 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:02:56,900 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:02:56,900 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 13 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 13 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=77} 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Evaluating tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 for canCommit - preceding participating shard names [cars] 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Skipping pending transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 in state CAN_COMMIT_COMPLETE 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Not moving tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 - cannot proceed with canCommit 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit - successfulFromPrevious: [] 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-1511764914$t] 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 candidate ready 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - candidate rootPath: / 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit - messages: [] 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 prepared 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,902 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 validated 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: success 03:02:56,903 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 03:02:56,903 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:02:56,903 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit returning result: true 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:56,903 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}]]] 03:02:56,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 submitted to persistence 03:02:56,903 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 26.85 μs 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}]]] 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=184} 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=77} 03:02:56,904 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,905 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 03:02:56,905 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=184} 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit - successfulFromPrevious: [] 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-106098026$u] 03:02:56,906 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: commit succeeded 03:02:56,906 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 03:02:56,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:02:56,906 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 109.1 μs 03:02:56,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 03:02:56,907 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 commit 03:02:56,907 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 finish commit 03:02:56,907 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,907 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914)] 03:02:56,907 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,907 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-106098026)] 03:02:56,907 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,907 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 27.57 μs 03:02:56,908 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=184}] 03:02:56,908 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 03:02:56,908 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 03:02:56,908 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:02:56,908 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 03:02:56,908 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 03:02:56,908 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=184} 03:02:56,908 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:02:56,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 03:02:56,909 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 03:02:56,910 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,910 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,910 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,910 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Forwarding CommitTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,912 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,912 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 144.5 μs 03:02:56,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 candidate ready 03:02:56,918 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - candidate rootPath: / 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - messages: [] 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 prepared 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 validated 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: success 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 candidate ready 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: canCommit - candidate rootPath: / 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: canCommit - messages: [] 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: preCommit - successfulFromPrevious: [] 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 prepared 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 scheduled for commit step 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 validated 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: success 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 candidate ready 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: canCommit - candidate rootPath: / 03:02:56,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: canCommit - messages: [] 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: preCommit - successfulFromPrevious: [] 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 prepared 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 scheduled for commit step 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 validated 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: success 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 candidate ready 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: canCommit - candidate rootPath: / 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: canCommit - messages: [] 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: preCommit - successfulFromPrevious: [] 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 prepared 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 scheduled for commit step 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 submitted to persistence 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 324 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 submitted to persistence 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 submitted to persistence 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 6, leaderSnapShotIndex: 1 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}], SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}]]] 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 submitted to persistence 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 candidate ready 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - candidate rootPath: / 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: canCommit - messages: [] 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 prepared 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,921 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}]]] 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 submitted to persistence 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 03:02:56,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 03:02:56,922 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 03:02:56,923 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 03:02:56,923 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}], SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}]]] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 4 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=324} 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=236} 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 5 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=6, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}]]] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=222} 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:02:56,927 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,928 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=236}] 03:02:56,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 6, nextIndex: 7 03:02:56,928 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=324}] 03:02:56,928 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=Simple{size=236}] 03:02:56,928 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=236}] 03:02:56,928 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=Simple{size=222}] 03:02:56,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 2 03:02:56,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 03:02:56,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 03:02:56,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=236} 03:02:56,928 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit - successfulFromPrevious: [] 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23-1511764914$w] 03:02:56,929 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=324} 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0: commit - successfulFromPrevious: [] 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$p] 03:02:56,929 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 157.0 μs 03:02:56,929 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,929 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 43.10 μs 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data Simple{size=236} 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0: commit - successfulFromPrevious: [] 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,929 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$q] 03:02:56,930 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 3, nextIndex: 4 03:02:56,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 modifications completed with CommitTransactionReply [version=13] 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=222} 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit - successfulFromPrevious: [] 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-106098026$x] 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 03:02:56,930 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 137.5 μs 03:02:56,930 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: commit succeeded 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:02:56,930 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,930 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 03:02:56,930 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 56.74 μs 03:02:56,930 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 canCommit 03:02:56,930 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 successfully completed direct commit 03:02:56,930 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 canCommit 03:02:56,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 6 03:02:56,930 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 successfully completed direct commit 03:02:56,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 03:02:56,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=236} 03:02:56,930 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 canCommit 03:02:56,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0: commit - successfulFromPrevious: [] 03:02:56,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 changing state from COMMIT_PENDING to COMMITTED 03:02:56,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry_shard-cars-member-2%3Adatastore-testTransactionForwardedToLeaderAfterRetry%400%3A5_15%23-779559026$r] 03:02:56,931 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 successfully completed direct commit 03:02:56,931 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 modifications completed with CommitTransactionReply [version=13] 03:02:56,931 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 03:02:56,931 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:56,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 03:02:56,931 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 58.75 μs 03:02:56,931 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 03:02:56,931 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,931 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 3 03:02:56,931 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 03:02:56,931 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 03:02:56,931 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 03:02:56,931 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=222} 03:02:56,932 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,932 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,932 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1511764914] 03:02:56,932 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,932 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 03:02:56,932 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 03:02:56,932 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 95.99 μs 03:02:56,932 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1196899748] 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 6 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 2, lastIndex(): 6 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data Simple{size=236} 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 03:02:56,933 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=324} 03:02:56,934 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,934 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 81.70 μs 03:02:56,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:02:56,934 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 03:02:56,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 03:02:56,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data Simple{size=236} 03:02:56,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:02:56,934 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 87.13 μs 03:02:56,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 6 03:02:56,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 03:02:56,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=236} 03:02:56,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:02:56,935 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 03:02:56,935 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 59.91 μs 03:02:56,935 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 03:02:56,935 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:02:56,935 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 52.58 μs 03:02:56,935 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,935 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-341306157] 03:02:56,936 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,936 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:56,936 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:56,936 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 03:02:56,936 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:02:56,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:02:56,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars 03:02:56,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 03:02:56,940 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,941 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 03:02:56,941 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:56,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 03:02:56,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 03:02:56,942 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,942 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#738879645] 03:02:56,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 03:02:56,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 03:02:56,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:02:56,943 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=7, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,944 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#738879645, transactionId=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] 03:02:56,944 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:02:56,944 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#738879645)] 03:02:56,944 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,948 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ReadData operation succeeded 03:02:56,949 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-522483371] 03:02:56,949 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:02:56,949 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 03:02:56,949 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:02:56,949 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 7, nextIndex: 8 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 5 03:02:56,949 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people 03:02:56,949 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - trying create transaction 03:02:56,949 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry)] 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 7 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=7933428} is already disabled 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 6 for term 5 03:02:56,949 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=7 03:02:56,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 03:02:56,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:56,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 03:02:56,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 4 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 03:02:56,951 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Creating transaction : member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:02:56,952 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#-2046412961] 03:02:56,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,953 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 03:02:56,953 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:02:56,953 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 03:02:56,953 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,953 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,953 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:02:56,953 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:56,953 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteTransactionContextSupport - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#-2046412961, transactionId=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=13] 03:02:56,953 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 03:02:56,953 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#-2046412961)] 03:02:56,954 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=3898597} is already disabled 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:02:56,955 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 03:02:56,956 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 03:02:56,957 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ReadData operation succeeded 03:02:56,957 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,957 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,958 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,958 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 03:02:56,958 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 03:02:56,958 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 03:02:56,958 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:56,958 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:02:56,958 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 03:02:56,958 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:56,958 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:02:56,958 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 03:02:56,958 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 03:02:56,959 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 03:02:56,959 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,959 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:02:56,959 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 03:02:56,959 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,959 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:56,960 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:56,960 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:56,965 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:56,965 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:56,965 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:56,965 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:56,982 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection closed with error: Connection reset 03:02:57,10 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 03:02:57,12 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:02:57,12 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-213: Closing 03:02:57,12 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 03:02:57,13 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:02:57,13 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-214: Closing 03:02:57,13 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 03:02:57,13 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 03:02:57,14 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:57,14 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:57,300 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:57,980 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1230194702964618941] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:58,30 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [709163680735081775] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:02:58,37 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:58,37 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:58,39 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:58.040] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:58,56 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:58,57 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:58,60 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:02:58.062] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:58,98 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:58,110 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4708882539997837002] 03:02:58,110 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:02:58,114 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:02:58,114 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:02:58,118 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:58,118 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:58,121 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:02:58,121 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:02:58,121 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:02:58,130 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:02:58,152 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1848196302606485074] 03:02:58,152 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:02:58,158 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:58,158 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:02:58,158 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:58,158 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:58,188 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 closeTransaction called 03:02:58,188 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-3:datastore-testLeadershipTransferOnShutdown@0:0_14#-354304086)] 03:02:58,188 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started [ERROR] [03/07/2024 15:02:58.190] [com.google.common.base.internal.Finalizer] [akka://cluster-test@127.0.0.1:2559/] swallowing exception during message send java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.remote.artery.Association.setupStopQuarantinedTimer(Association.scala:633) at akka.remote.artery.Association.send(Association.scala:399) at akka.remote.artery.ArteryTransport.send(ArteryTransport.scala:707) at akka.remote.RemoteActorRef.$bang(RemoteActorRefProvider.scala:737) at akka.actor.ActorRef.tell(ActorRef.scala:131) at akka.actor.ActorSelection$.rec$1(ActorSelection.scala:287) at akka.actor.ActorSelection$.deliverSelection(ActorSelection.scala:291) at akka.actor.ActorSelection.tell(ActorSelection.scala:50) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.sendOperationAsync(ActorUtils.java:398) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContext.closeTransaction(RemoteTransactionContext.java:84) at org.opendaylight.controller.cluster.datastore.TransactionContextCleanup.finalizeReferent(TransactionContextCleanup.java:60) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.google.common.base.internal.Finalizer.finalizeReference(Finalizer.java:208) at com.google.common.base.internal.Finalizer.cleanUp(Finalizer.java:168) at com.google.common.base.internal.Finalizer.run(Finalizer.java:146) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:02:58,190 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 closeTransaction called 03:02:58,190 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-cars-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_16#738879645)] 03:02:58,191 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 closeTransaction called 03:02:58,191 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2559/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-people-member-1:datastore-testTransactionForwardedToLeaderAfterRetry@0:0_17#-2046412961)] 03:02:58,211 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6021682321798077047] 03:02:58,211 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:02:58,218 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:02:58,218 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:02:58,223 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:02:58,223 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:02:58,228 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:58,229 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:58,231 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,239 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#417823373] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:58,240 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,252 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 03:02:58,252 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:02:58,252 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@27e8a4af to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@16249ada 03:02:58,252 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5435c424 03:02:58,252 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709823778252, None)) 03:02:58,254 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823778251,0,0)) 03:02:58,254 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@16249ada to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5c97d52d 03:02:58,254 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 03:02:58,255 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:58,258 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:58,258 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,258 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,258 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:58,259 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,259 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@575ab3ec 03:02:58,259 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:58,259 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,261 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,275 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,275 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1968131965] 03:02:58,276 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 03:02:58,277 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 03:02:58,278 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1968131965] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:58,278 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#1232269972] 03:02:58,284 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,285 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-336622018] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:58,290 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#1232269972] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:58,305 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1053199888 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,305 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1053199888] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:58,306 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:58,307 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:58,307 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:02:58,307 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:02:58,307 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 03:02:58,307 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 03:02:58,307 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 03:02:58,307 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,310 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,310 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,311 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 03:02:58,311 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:02:58,311 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@56f28222 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@60f82b3f 03:02:58,311 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@60229df6 03:02:58,312 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709823778312, None)) 03:02:58,316 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:02:58,316 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823778311,0,0)) 03:02:58,317 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@34f394f7 03:02:58,317 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@60f82b3f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@37cdf07f 03:02:58,317 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 03:02:58,317 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:58,317 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,321 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:58,339 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 03:02:58,341 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:58,341 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,341 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,342 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 03:02:58,343 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 03:02:58,343 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#580181223] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:58,347 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-742022696] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:58,351 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:58,351 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:58,351 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:02:58,351 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:02:58,351 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,351 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 03:02:58,351 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 03:02:58,354 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-742022696 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,354 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 03:02:58,354 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,356 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-1654646222] 03:02:58,357 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-1654646222] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:58,368 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#417823373] 03:02:58,368 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 03:02:58,368 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1968131965] 03:02:58,371 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 03:02:58,379 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:02:58,382 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:02:58,616 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:02:58,616 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 03:02:58,616 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:02:58,616 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,616 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 03:02:58,616 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:02:58,617 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 03:02:58,617 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:02:58,617 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:02:58,617 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 03:02:58,617 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 03:02:58,873 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:02:58,936 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 03:02:58,936 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:58,936 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 03:02:59,138 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:02:59,138 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:02:59,138 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:59,138 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,138 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,138 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:59,138 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,138 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,138 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,138 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,147 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:59,147 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,147 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,147 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:59,147 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,147 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,147 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,222 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 03:02:59,222 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,222 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 03:02:59,222 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,222 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:59,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 03:02:59,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:02:59,224 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:02:59,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=null 03:02:59,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:02:59,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,224 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:02:59,225 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1968131965] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:59,232 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:02:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:02:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:02:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:02:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:59,236 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:02:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 03:02:59,236 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 03:02:59,237 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 03:02:59,239 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:02:59,239 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:59,239 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:59,239 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 03:02:59,239 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:02:59,239 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:02:59,239 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,239 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 03:02:59,240 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1968131965] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:59,240 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:02:59,240 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:59,240 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:59,240 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:02:59,245 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 03:02:59,245 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:02:59,276 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:59,279 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:02:59,283 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,288 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1824707636] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:59,290 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,292 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 03:02:59,292 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:02:59,292 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2f8e4c78 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2149a7f9 03:02:59,292 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@25762708 03:02:59,292 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709823779292, None)) 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,293 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823779291,0,0)) 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,293 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2149a7f9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4a6b136 03:02:59,293 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@52b0bc42 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:02:59,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,293 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 03:02:59,294 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-336622018] 03:02:59,294 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Queuing TransactionOperation 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1571248624] 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1571248624] 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,294 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1571248624] 03:02:59,295 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 03:02:59,295 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 03:02:59,295 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1571248624] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:59,295 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1952109658 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,296 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1952109658] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:59,297 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:02:59,297 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 03:02:59,297 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 03:02:59,297 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars 03:02:59,297 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 03:02:59,297 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 03:02:59,298 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:02:59,298 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:02:59,298 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 03:02:59,298 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 03:02:59,298 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:02:59,298 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:02:59,298 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,298 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, disabling transaction tracking 03:02:59,304 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 03:02:59,304 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 03:02:59,304 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 03:02:59,304 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 03:02:59,305 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-319940586] 03:02:59,305 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-319940586] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:59,307 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-319940586, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0, version=13] 03:02:59,307 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:02:59,307 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 3 in "AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 03:02:59,307 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-319940586)] 03:02:59,307 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=null 03:02:59,307 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:02:59,307 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,308 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 03:02:59,308 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 03:02:59,308 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:59,308 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 03:02:59,308 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 03:02:59,308 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 03:02:59,308 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:59,308 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:02:59,308 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:59,308 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:59,309 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:02:59,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 03:02:59,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:59,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,309 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 03:02:59,310 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 ReadData operation succeeded 03:02:59,311 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 03:02:59,311 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:02:59,312 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 03:02:59,312 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 03:02:59,312 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,312 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-219: Closing 03:02:59,312 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 03:02:59,312 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-1627482351] 03:02:59,312 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,312 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-1627482351] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:02:59,318 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:02:59,318 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:02:59,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 03:02:59,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 03:02:59,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:02:59,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 03:02:59,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 03:02:59,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 03:02:59,320 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:59,322 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:59,322 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:59,322 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:59,322 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:02:59,325 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:02:59,328 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 03:02:59,328 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 03:02:59,337 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,338 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,338 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 03:02:59,338 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:02:59,338 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,338 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 03:02:59,338 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 03:02:59,338 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:02:59,340 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionRetryWithInitialAskTimeoutExOnCreateTx commit rateLimit adjusted to 85.79685415538616 03:02:59,340 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:02:59,341 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars, trying to use DirectTransactionContextWrapper 03:02:59,341 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:02:59,341 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 03:02:59,341 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 03:02:59,341 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 03:02:59,341 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying 1 components for commit 03:02:59,341 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying transaction for shard cars 03:02:59,341 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 03:02:59,341 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 canCommit 03:02:59,854 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:02:59,883 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:02:59,883 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 03:02:59,883 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 03:02:59,883 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 03:02:59,883 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 03:02:59,883 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 03:02:59,883 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 03:02:59,883 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:02:59,883 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,884 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 03:02:59,884 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=3] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,884 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,884 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 4 03:02:59,884 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 03:02:59,884 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:02:59,884 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:02:59,884 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:02:59,885 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 03:02:59,885 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 03:02:59,885 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:59,885 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:59,885 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 03:02:59,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:02:59,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 03:02:59,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:02:59,887 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 3 - updating term 03:02:59,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 03:02:59,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=3] - currentTerm: 4, votedFor: null, lastIndex: 0, lastTerm: 3 03:02:59,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:02:59,889 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:02:59,889 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:02:59,889 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 03:02:59,893 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:02:59,893 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:59,894 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:02:59,895 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:02:59,895 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,895 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 03:02:59,895 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 4 03:02:59,895 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 03:02:59,895 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 03:02:59,895 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to PreLeader 03:02:59,895 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:02:59,895 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:02:59,895 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Replicate message: identifier: null, logIndex: 1, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 03:02:59,895 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:59,895 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:59,896 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:02:59,896 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:59,896 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:59,896 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to PreLeader 03:02:59,896 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:02:59,896 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:02:59,896 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:02:59,896 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:02:59,897 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 03:02:59,897 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 3 does not match the current term 4 03:02:59,897 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:59,897 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,897 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:02:59,898 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 03:02:59,898 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:02:59,898 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 3 does not match the current term 4 03:02:59,898 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 3 does not match the current term 4 03:02:59,898 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:02:59,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=3, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:02:59,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 03:02:59,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:02:59,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:02:59,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:02:59,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 2 03:02:59,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:02:59,900 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 1, nextIndex: 2 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 3 does not match the current term 4 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Setting last applied to 0 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: leader state for ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} not found 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Setting last applied to 1 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Received ApplyState [identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, replicatedLogEntry=SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]] - lastApplied: 1, lastIndex: 1 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 4 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader)_slicer-id-222: Closing 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 03:02:59,901 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 03:02:59,902 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from PreLeader to Leader 03:02:59,902 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from PreLeader to Leader 03:02:59,902 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:02:59,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Commit index set to 1 03:02:59,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 03:02:59,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 0 03:02:59,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 03:02:59,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 03:02:59,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:02:59,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 03:02:59,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 1 03:02:59,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 03:02:59,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:02:59,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 03:03:00,331 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4708882539997837002] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:00,334 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4708882539997837002] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:00,405 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:03:00,434 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:03:00,434 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-336622018] 03:03:00,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:00,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#580181223, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4a6af477] 03:03:00,434 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#580181223, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4a6af477] 03:03:00,434 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#580181223)] found - trying create transaction 03:03:00,434 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#580181223)] 03:03:00,435 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,435 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#-11825340] 03:03:00,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#-11825340, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, version=13] 03:03:00,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:00,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 directCommit called 03:03:00,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 sending 1 batched modifications, ready: true 03:03:00,435 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#-11825340] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:00,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_19#-11825340)] 03:03:00,436 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: handleForwardedReadyTransaction for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, client version 13 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 ready 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 validated 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: canCommit for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: success 03:03:00,436 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 candidate ready 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - candidate rootPath: / 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - messages: [] 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 prepared 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Finishing commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 191 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=191}] 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=191}] 03:03:00,437 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:00,438 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:00,438 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:00,438 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:03:00,438 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=191}]]] 03:03:00,438 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 submitted to persistence 03:03:00,439 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=191}] 03:03:00,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=191}]]] 03:03:00,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 03:03:00,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:00,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=191} 03:03:00,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=191}] 03:03:00,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 3 03:03:00,440 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:00,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=191}] 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 2, nextIndex: 3 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 2 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 1 to 2 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 2 data Simple{size=191} 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: commit - successfulFromPrevious: [] 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:03:00,441 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_shard-cars-member-2%3Adatastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx%400%3A1_19%23-11825340$d] 03:03:00,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-1333585719] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 03:03:00,442 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 successfully completed direct commit 03:03:00,442 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=13] 03:03:00,442 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:03:00,442 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-1333585719] 03:03:00,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:00,442 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=2 03:03:00,442 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 03:03:00,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:03:00,442 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 03:03:00,442 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 03:03:00,442 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 123.3 μs 03:03:00,442 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#-1333585719] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:00,443 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:03:00,443 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 03:03:00,443 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 03:03:00,445 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#580181223] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1571248624] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1571248624] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:00,445 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 03:03:00,445 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:03:00,446 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:03:00,446 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:03:00,446 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:03:00,446 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:03:00,446 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:03:00,446 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:03:00,447 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:03:00,447 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 03:03:00,447 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1824707636] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1824707636] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:00,447 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:03:00,447 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1824707636] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1824707636] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:00,447 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#552378692] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#552378692] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:00,448 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:03:00,448 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:03:00,448 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-223: Closing 03:03:00,448 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 03:03:00,448 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:03:00,449 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:00,450 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:01,462 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1848196302606485074] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/07/2024 15:03:01.467] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-173)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:01.467] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-173)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:01.472] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:01,485 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:01,485 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:01,488 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:01.489] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:01,512 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:01,520 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6825916730155904455] 03:03:01,520 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:01,534 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:01,534 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:01,534 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:01,534 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:01,534 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:01,534 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:01,534 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:01,544 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:01,556 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4519353748460754907] 03:03:01,556 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:01,558 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:01,558 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:01,559 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:01,559 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:01,592 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:01,606 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8333903393686331437] 03:03:01,606 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:01,606 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:01,606 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:01,609 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:01,609 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:01,621 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:01,623 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:01,626 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 03:03:01,666 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 03:03:01,667 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-733215729] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,673 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:01,673 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:01,689 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 03:03:01,689 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:03:01,689 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3448cd6c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5ea2650a 03:03:01,689 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 03:03:01,689 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,690 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,690 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:01,690 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,690 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,710 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5b65ef6 03:03:01,710 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1709823781691, None)) 03:03:01,710 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823781690,0,0)) 03:03:01,710 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5ea2650a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@54673223 03:03:01,710 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6d6fa145 03:03:01,710 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 03:03:01,711 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:01,713 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:01,715 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 03:03:01,716 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:01,716 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,723 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 03:03:01,728 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-475563644] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,728 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,728 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,728 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333] 03:03:01,728 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,729 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,729 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,729 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1737905920] 03:03:01,729 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 03:03:01,729 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,730 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-988551424 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,730 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#897238494] 03:03:01,730 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-988551424] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,730 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#897238494] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,730 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 03:03:01,732 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 03:03:01,733 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 03:03:01,734 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1737905920] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,734 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1587024349] 03:03:01,734 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#1817812703] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,737 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:01,737 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:01,737 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 03:03:01,737 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 03:03:01,738 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:01,743 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:01,749 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:01,749 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:01,749 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 03:03:01,749 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 03:03:01,750 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#1817812703 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,752 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1587024349] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,753 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:01,753 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 03:03:01,753 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 03:03:01,753 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 03:03:01,754 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 03:03:01,757 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:01,757 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 03:03:01,757 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 03:03:01,757 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 03:03:01,757 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 03:03:01,757 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 03:03:01,757 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,758 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,758 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:01,758 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,758 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,758 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 03:03:01,758 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:03:01,758 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@70234f49 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a903ec8 03:03:01,758 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@49c8b570 03:03:01,759 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1709823781759, None)) 03:03:01,760 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823781758,0,0)) 03:03:01,760 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6640111f 03:03:01,760 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a903ec8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@746d53db 03:03:01,760 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 03:03:01,760 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:01,760 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,761 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,761 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,761 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#63096646] 03:03:01,761 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,761 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,761 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,761 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 03:03:01,761 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-2-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-624850858] 03:03:01,762 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 03:03:01,762 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 03:03:01,762 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 03:03:01,763 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1786984848 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,764 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#63096646] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,764 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-624850858] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,764 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1786984848] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,764 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1899982810] 03:03:01,764 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-954881188] 03:03:01,764 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1899982810] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,764 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-954881188] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,768 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:01,768 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:01,768 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:01,768 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:01,768 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 03:03:01,768 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 03:03:01,768 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 03:03:01,768 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 03:03:01,768 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:01,768 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 03:03:01,768 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:01,768 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:01,768 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 03:03:01,768 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,768 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#709816229 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:01,768 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 03:03:01,768 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 03:03:01,768 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#709816229] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:01,769 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 03:03:01,769 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 03:03:01,769 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 03:03:01,769 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 03:03:01,811 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-733215729] 03:03:01,811 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 03:03:01,811 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333] 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 03:03:02,38 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 03:03:02,38 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:02,58 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 03:03:02,59 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 03:03:02,59 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 03:03:02,284 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:02,288 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:02,288 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,288 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 03:03:02,298 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:02,298 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,298 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 03:03:02,304 PM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:02,539 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:02,539 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,540 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,547 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:02,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,568 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 03:03:02,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:02,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 03:03:02,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:02,570 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:02,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=null 03:03:02,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:02,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:03:02,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:03:02,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:03:02,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 03:03:02,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:02,572 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:03:02,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:03:02,572 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 03:03:02,572 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 03:03:02,573 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:03:02,573 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:02,573 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:02,573 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 03:03:02,573 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 03:03:02,573 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,573 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 03:03:02,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:03:02,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:02,574 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:03:02,574 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:02,574 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:02,588 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:02,588 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,588 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 03:03:02,588 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,590 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:02,590 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 03:03:02,590 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:02,590 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:02,590 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=null 03:03:02,590 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:02,590 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,590 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 03:03:02,593 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 03:03:02,593 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 03:03:02,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 03:03:02,595 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,595 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 03:03:02,596 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:03:02,596 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:02,596 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:02,596 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 03:03:02,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:03:02,596 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:02,623 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:02,666 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-733215729] 03:03:02,666 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 03:03:02,667 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1737905920] 03:03:02,667 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testReadWriteTransactionWithMultipleShards commit rateLimit adjusted to 85.79685415538616 03:03:02,667 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,668 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-475563644] 03:03:02,668 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:03:02,668 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-475563644] 03:03:02,668 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:03:02,668 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 03:03:02,668 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 03:03:02,668 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:03:02,668 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 03:03:02,668 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:03:02,668 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 03:03:02,669 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:02,669 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 03:03:02,669 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:03:02,669 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 03:03:02,669 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 03:03:02,669 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 03:03:02,669 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars 03:03:02,670 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 03:03:02,670 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people 03:03:02,670 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 03:03:02,670 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 03:03:02,670 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 03:03:02,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 03:03:02,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,672 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,672 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 03:03:02,673 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:02,673 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,673 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,673 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:02,673 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,673 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:02,673 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 03:03:02,673 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:02,673 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,673 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,673 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 03:03:02,673 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,673 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-510505953] 03:03:02,673 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-510505953] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:02,674 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#1289573370] 03:03:02,674 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#1289573370] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:02,674 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,674 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-510505953, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] 03:03:02,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:02,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:02,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:02,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 03:03:02,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_20#-510505953)] 03:03:02,675 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 03:03:02,675 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:02,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 03:03:02,675 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:02,675 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:02,675 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 03:03:02,675 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#1289573370, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] 03:03:02,675 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:03:02,675 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 03:03:02,675 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_21#1289573370)] 03:03:02,676 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,676 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 03:03:02,677 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 03:03:02,677 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=5547899} is already disabled 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 03:03:02,678 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=5879477} is already disabled 03:03:02,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:02,678 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,679 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:02,679 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,679 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 13 03:03:02,679 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,679 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,679 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 03:03:02,679 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 13 03:03:02,679 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,679 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333)] resolved 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1737905920)] resolved 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 03:03:02,680 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 03:03:02,680 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:02,680 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 03:03:02,680 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333)] 03:03:02,680 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 03:03:02,680 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333)] 03:03:02,680 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:03:02,680 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:03:02,680 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:02,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 03:03:02,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:02,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 03:03:02,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:03:02,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 03:03:02,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:03:02,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:03:02,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:02,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 03:03:02,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:02,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 03:03:02,684 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:03:02,684 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1737905920)] 03:03:02,684 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1737905920)] 03:03:02,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 03:03:02,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:02,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 03:03:02,687 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:03:02,687 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 03:03:02,687 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 03:03:02,687 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 03:03:02,687 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333)] 03:03:02,687 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333)] 03:03:02,687 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1737905920)] 03:03:02,687 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#1737905920)] 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,688 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}]]] 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=183}] 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=183}] 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:02,689 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}]]] 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:02,689 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,689 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:02,690 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:02,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=183}]]] 03:03:02,690 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 03:03:02,690 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 03:03:02,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 03:03:02,690 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 03:03:02,690 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 03:03:02,690 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:02,690 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=183}] 03:03:02,690 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 03:03:02,691 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=183}]]] 03:03:02,691 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:02,691 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:02,691 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=183} 03:03:02,691 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=183}] 03:03:02,691 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 03:03:02,691 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:02,693 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=183}] 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-cars-testReadWriteTransactionWithMultipleShards%23-1365230333$g] 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-276465112] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 03:03:02,693 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-276465112] 03:03:02,693 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:03:02,693 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:02,693 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 03:03:02,693 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:03:02,693 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 3 03:03:02,693 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,693 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:03:02,693 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:03:02,693 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:02,693 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:02,694 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-people-testReadWriteTransactionWithMultipleShards%231737905920$h] 03:03:02,694 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:02,694 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 108.0 μs 03:03:02,694 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1983033024] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 03:03:02,694 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-276465112] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:02,694 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1983033024] 03:03:02,694 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 3 03:03:02,694 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:03:02,694 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:02,694 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 47.00 μs 03:03:02,694 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:02,696 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 03:03:02,696 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:02,696 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 03:03:02,696 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:03:02,697 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 03:03:02,697 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,697 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 03:03:02,697 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1596489843] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 03:03:02,697 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:02,697 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 03:03:02,697 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:03:02,698 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 03:03:02,698 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:02,698 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 3 03:03:02,698 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:02,698 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1046298571] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 03:03:02,698 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 3 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1046298571] 03:03:02,698 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1596489843] 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 110.5 μs 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 87.23 μs 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1046298571] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:02,698 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1596489843] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:02,698 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:03:02,699 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:02,699 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:02,699 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 03:03:02,699 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 03:03:02,699 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 03:03:02,700 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:03:02,701 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#1347698117] 03:03:02,701 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#1347698117, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 03:03:02,701 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:02,701 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#1347698117)] 03:03:02,703 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 03:03:02,703 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 03:03:02,703 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:02,703 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 03:03:02,703 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 03:03:02,704 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 03:03:02,704 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:03:02,704 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#1675289244] 03:03:02,705 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#1675289244, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 03:03:02,705 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:03:02,705 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#1675289244)] 03:03:02,709 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 03:03:02,709 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 03:03:02,710 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 03:03:02,710 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 03:03:02,710 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c04cb0c 03:03:02,710 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c04cb0c 03:03:02,710 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:02,710 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 03:03:02,710 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:02,710 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:02,710 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3bc8c278 03:03:02,710 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3bc8c278 03:03:02,710 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:02,710 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 03:03:02,710 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:02,711 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards] 03:03:02,711 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards] 03:03:02,711 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:02,711 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:02,711 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:02,711 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:02,711 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:02,711 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:02,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:03:02,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:03:02,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:02,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 03:03:02,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:03:02,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:03:02,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:02,712 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 03:03:02,932 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 218, lastApplied : 1, commitIndex : 1 03:03:03,127 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 03:03:03,128 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1365230333]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:03,128 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:03,128 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-230: Closing 03:03:03,128 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:03:03,128 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:03:03,128 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-231: Closing 03:03:03,128 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:03:03,128 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 03:03:03,128 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 03:03:03,129 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:03:03,129 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 03:03:03,129 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 03:03:03,129 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 03:03:03,130 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:03,130 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:03,135 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:03,135 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:03,135 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:03,135 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:03,137 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:03,150 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:03:03,150 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 03:03:03,150 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:03:03,150 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 03:03:03,150 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:03:03,150 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:03:03,151 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 03:03:03,155 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 03:03:03,156 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:03,156 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:04,142 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6825916730155904455] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:04,172 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4519353748460754907] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:04,179 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:04,179 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:04,179 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:04.182] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:04,196 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:04,196 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:04,199 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:04.200] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:04,234 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:04,247 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5950622864267542999] 03:03:04,247 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:04,249 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:04,249 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:04,256 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:04,256 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:04,267 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:04,267 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:04,267 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:04,269 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:03:04,284 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [966420734259826006] 03:03:04,285 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:04,293 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:04,293 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:04,308 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:04,308 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:04,311 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:04,321 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [913862124478781127] 03:03:04,321 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:04,329 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:04,329 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:04,332 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:04,332 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:04,343 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 03:03:04,343 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:04,343 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_22#1347698117)] 03:03:04,343 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 closeTransaction called 03:03:04,343 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_18#-319940586)] 03:03:04,343 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [ERROR] [03/07/2024 15:03:04.343] [com.google.common.base.internal.Finalizer] [akka://cluster-test@127.0.0.1:2558/] swallowing exception during message send java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.remote.artery.Association.setupStopQuarantinedTimer(Association.scala:633) at akka.remote.artery.Association.send(Association.scala:399) at akka.remote.artery.ArteryTransport.send(ArteryTransport.scala:707) at akka.remote.RemoteActorRef.$bang(RemoteActorRefProvider.scala:737) at akka.actor.ActorRef.tell(ActorRef.scala:131) at akka.actor.ActorSelection$.rec$1(ActorSelection.scala:287) at akka.actor.ActorSelection$.deliverSelection(ActorSelection.scala:291) at akka.actor.ActorSelection.tell(ActorSelection.scala:50) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.sendOperationAsync(ActorUtils.java:398) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContext.closeTransaction(RemoteTransactionContext.java:84) at org.opendaylight.controller.cluster.datastore.TransactionContextCleanup.finalizeReferent(TransactionContextCleanup.java:60) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.google.common.base.internal.Finalizer.finalizeReference(Finalizer.java:208) at com.google.common.base.internal.Finalizer.cleanUp(Finalizer.java:181) at com.google.common.base.internal.Finalizer.run(Finalizer.java:146) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:03:04,344 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 03:03:04,344 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_23#1675289244)] 03:03:04,353 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 03:03:04,358 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 03:03:04,358 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 03:03:04,362 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:04,362 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:04,364 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:04.366] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:04,375 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:04,375 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:04,377 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:04.378] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:04,413 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:04,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4501509045548228290] 03:03:04,429 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:04,431 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:04,431 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:04,433 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:04,433 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:04,449 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:04,449 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:04,449 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:04,451 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:04,457 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-595773636353726061] 03:03:04,457 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:04,459 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:04,459 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:04,460 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:04,460 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:04,473 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:04,501 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1143809647862410007] 03:03:04,501 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:04,504 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:04,504 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:04,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:04,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:04,519 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:04,520 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:04,523 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 03:03:04,551 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 03:03:04,551 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:03:04,551 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@39efca65 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4004d869 03:03:04,551 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7a57595b 03:03:04,552 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1023616608] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:04,552 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 03:03:04,552 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1709823784552, None)) 03:03:04,553 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823784551,0,0)) 03:03:04,553 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4004d869 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@45b155c2 03:03:04,553 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 03:03:04,554 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:04,556 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:04,558 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 03:03:04,560 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 03:03:04,560 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,560 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:04,560 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@235aed66 03:03:04,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:04,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,561 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,561 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1377656660] 03:03:04,561 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 03:03:04,563 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 03:03:04,563 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1377656660] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:04,564 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-957580981 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,564 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-957580981] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:04,564 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#1222424354] 03:03:04,566 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#1222424354] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:04,566 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:04,566 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:04,567 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 03:03:04,567 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 03:03:04,567 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 03:03:04,567 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 03:03:04,573 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 03:03:04,573 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 03:03:04,573 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 03:03:04,576 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#546894271] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:04,577 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 03:03:04,580 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 03:03:04,580 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 03:03:04,580 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,580 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:03:04,581 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3658419f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3e8072b7 03:03:04,581 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4e77b2b6 03:03:04,581 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1709823784581, None)) 03:03:04,581 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823784580,0,0)) 03:03:04,581 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4a231295 03:03:04,581 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3e8072b7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@dde0fd7 03:03:04,581 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 03:03:04,582 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:04,582 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,582 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 03:03:04,583 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 03:03:04,583 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#-470414713] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:04,583 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-373776075] 03:03:04,583 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-112609023 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,583 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-373776075] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:04,583 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-112609023] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:04,584 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:04,584 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:04,584 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 03:03:04,584 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 03:03:04,584 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 03:03:04,584 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 03:03:04,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 03:03:04,584 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 03:03:04,585 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 03:03:04,605 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:04,608 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:04,608 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:04,608 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,608 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,608 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,609 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:04,613 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:04,632 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1023616608] 03:03:04,632 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 03:03:04,632 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1377656660] 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 03:03:04,796 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 03:03:04,796 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 03:03:05,55 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 03:03:05,55 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,56 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 03:03:05,186 PM [cluster-test-shard-dispatcher-20] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:05,296 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 03:03:05,296 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,296 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 03:03:05,447 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:05,447 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:05,448 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:05,448 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,448 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:05,448 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,448 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,448 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,506 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:05,506 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,506 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:05,506 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,506 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,523 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:05,529 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 03:03:05,529 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,529 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 03:03:05,529 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:05,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 03:03:05,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:05,534 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:05,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 03:03:05,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:05,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,534 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:05,540 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:05,540 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:05,540 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 03:03:05,540 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:05,540 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:05,540 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 03:03:05,540 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 03:03:05,545 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 03:03:05,545 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 03:03:05,545 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:05,545 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:05,545 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 03:03:05,545 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 03:03:05,549 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 03:03:05,549 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,550 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 03:03:05,550 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 03:03:05,550 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:05,551 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 03:03:05,551 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:05,551 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:05,551 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 03:03:05,589 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleTransactionsWritesInQuickSuccession commit rateLimit adjusted to 85.79685415538616 03:03:05,589 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,589 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#546894271] 03:03:05,589 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:03:05,589 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:03:05,589 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying 1 components for commit 03:03:05,589 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 03:03:05,589 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:03:05,590 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 canCommit 03:03:05,590 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:05,590 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 03:03:05,590 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 03:03:05,590 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars 03:03:05,590 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 03:03:05,590 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:05,590 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:03:05,591 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 directCommit called 03:03:05,591 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 03:03:05,591 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,593 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, disabling transaction tracking 03:03:05,593 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 03:03:05,593 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 03:03:05,593 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:05,593 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:05,593 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,594 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 03:03:05,594 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 03:03:05,594 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 03:03:05,594 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@488ef05b 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 2 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 of 2 operations, client version 13 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 ready 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,595 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 validated 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: success 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 candidate ready 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - messages: [] 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 prepared 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,596 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 submitted to persistence 03:03:05,599 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 03:03:05,600 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 03:03:05,600 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 03:03:05,600 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 03:03:05,600 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:03:05,600 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:05,600 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 03:03:05,600 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 03:03:05,600 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 1 03:03:05,600 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 0, nextIndex: 1 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=7586689} is already disabled 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 03:03:05,601 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=183} 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 0 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:03:05,606 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 03:03:05,607 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 03:03:05,607 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 03:03:05,607 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 03:03:05,607 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 2, nextIndex: 3 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} has no callback 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 03:03:05,607 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,608 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 03:03:05,608 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:05,608 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:03:05,608 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$a] 03:03:05,608 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1821262494] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 03:03:05,608 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1821262494] 03:03:05,608 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,608 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1821262494] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:05,608 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 91.96 μs 03:03:05,608 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 03:03:05,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:05,609 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 03:03:05,609 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 successfully completed direct commit 03:03:05,609 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=13] 03:03:05,610 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,610 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,610 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,610 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 03:03:05,610 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}] 03:03:05,610 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying 1 components for commit 03:03:05,610 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 03:03:05,610 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 directCommit called 03:03:05,610 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: true 03:03:05,610 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,610 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 canCommit 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 2 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 03:03:05,613 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:03:05,613 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,613 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@4d41f7df 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 of 1 operations, client version 13 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 ready 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 validated 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: success 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,614 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-2121578913] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 03:03:05,614 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 03:03:05,614 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 candidate ready 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - messages: [] 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 prepared 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 03:03:05,614 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 03:03:05,615 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-2121578913] 03:03:05,615 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:05,615 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:03:05,615 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,615 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,615 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 03:03:05,615 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 submitted to persistence 03:03:05,615 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 119.1 μs 03:03:05,615 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,616 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 03:03:05,615 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-2121578913] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:05,621 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 03:03:05,621 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:03:05,621 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:03:05,621 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 03:03:05,621 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 03:03:05,622 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 03:03:05,622 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:05,622 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 03:03:05,622 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 03:03:05,622 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 03:03:05,622 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 03:03:05,622 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 03:03:05,622 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 03:03:05,622 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,622 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 03:03:05,622 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:03:05,623 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:03:05,623 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$b] 03:03:05,623 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 03:03:05,623 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:05,623 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 03:03:05,624 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 modifications completed with CommitTransactionReply [version=13] 03:03:05,624 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 successfully completed direct commit 03:03:05,624 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,624 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 03:03:05,624 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,624 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,624 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 03:03:05,624 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 136.5 μs 03:03:05,624 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,624 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 03:03:05,625 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 03:03:05,625 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,625 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2a021c59 03:03:05,625 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 03:03:05,625 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 03:03:05,625 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 03:03:05,625 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 03:03:05,625 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 03:03:05,625 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#196597166] 03:03:05,626 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#196597166] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:05,626 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:03:05,626 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,626 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 03:03:05,626 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 03:03:05,626 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 81.02 μs 03:03:05,627 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#196597166, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0, version=13] 03:03:05,628 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:05,628 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#196597166)] 03:03:05,631 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 ReadData operation succeeded 03:03:05,631 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 closeTransaction called 03:03:05,631 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_24#196597166)] 03:03:05,631 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,631 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,631 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,631 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 03:03:05,632 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 03:03:05,632 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying 1 components for commit 03:03:05,632 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying transaction for shard cars 03:03:05,632 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 directCommit called 03:03:05,632 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 sending 1 batched modifications, ready: true 03:03:05,632 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,632 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 canCommit 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2a021c59 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 of 1 operations, client version 13 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 ready 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 validated 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: success 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 candidate ready 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - candidate rootPath: / 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - messages: [] 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: preCommit - successfulFromPrevious: [] 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 prepared 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,634 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,635 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 03:03:05,635 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 03:03:05,635 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 03:03:05,635 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:05,635 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 03:03:05,635 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,635 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 03:03:05,635 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 submitted to persistence 03:03:05,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 03:03:05,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:03:05,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 03:03:05,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 03:03:05,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 03:03:05,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 03:03:05,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:05,638 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 03:03:05,639 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 4, nextIndex: 5 03:03:05,639 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 2 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: commit - successfulFromPrevious: [] 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from COMMIT_PENDING to COMMITTED 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$e] 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 03:03:05,640 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 03:03:05,640 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,641 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 79.02 μs 03:03:05,646 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 modifications completed with CommitTransactionReply [version=13] 03:03:05,646 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 successfully completed direct commit 03:03:05,646 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 03:03:05,646 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,646 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,646 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 03:03:05,646 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,647 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Queuing TransactionOperation 03:03:05,647 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 03:03:05,648 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,648 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5ed0f4ea 03:03:05,648 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-179727673] 03:03:05,648 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-179727673] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:05,648 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-179727673, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0, version=13] 03:03:05,649 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:05,649 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-179727673)] 03:03:05,651 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 ReadData operation succeeded 03:03:05,651 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 closeTransaction called 03:03:05,651 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_25#-179727673)] 03:03:05,651 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,651 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,651 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,651 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 03:03:05,652 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}] 03:03:05,652 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying 1 components for commit 03:03:05,652 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying transaction for shard cars 03:03:05,652 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 directCommit called 03:03:05,652 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 sending 1 batched modifications, ready: true 03:03:05,652 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,652 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 canCommit 03:03:05,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5ed0f4ea 03:03:05,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 of 1 operations, client version 13 03:03:05,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,653 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 ready 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 validated 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: success 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 candidate ready 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - candidate rootPath: / 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - messages: [] 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: preCommit - successfulFromPrevious: [] 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 prepared 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:05,654 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 03:03:05,655 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,655 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 03:03:05,655 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 submitted to persistence 03:03:05,655 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 4 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 5 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 03:03:05,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:03:05,657 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 03:03:05,657 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,658 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 81.09 μs 03:03:05,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 03:03:05,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 5, nextIndex: 6 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: commit - successfulFromPrevious: [] 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from COMMIT_PENDING to COMMITTED 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:03:05,658 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$h] 03:03:05,659 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 03:03:05,659 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 03:03:05,659 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 03:03:05,659 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,659 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 50.25 μs 03:03:05,660 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 successfully completed direct commit 03:03:05,660 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 modifications completed with CommitTransactionReply [version=13] 03:03:05,660 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 03:03:05,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 5 03:03:05,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 03:03:05,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 03:03:05,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 03:03:05,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 03:03:05,660 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,660 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,660 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 03:03:05,660 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:03:05,660 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Queuing TransactionOperation 03:03:05,660 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,660 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 45.92 μs 03:03:05,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 03:03:05,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 03:03:05,661 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 03:03:05,661 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,661 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@d1457a9 03:03:05,661 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#982800079] 03:03:05,661 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#982800079] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:05,662 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#982800079, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0, version=13] 03:03:05,662 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:05,662 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#982800079)] 03:03:05,664 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 ReadData operation succeeded 03:03:05,664 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 closeTransaction called 03:03:05,664 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_26#982800079)] 03:03:05,664 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,664 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,664 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,664 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 03:03:05,664 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}] 03:03:05,664 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying 1 components for commit 03:03:05,664 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying transaction for shard cars 03:03:05,664 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 directCommit called 03:03:05,664 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 sending 1 batched modifications, ready: true 03:03:05,664 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,664 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 canCommit 03:03:05,665 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,665 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@d1457a9 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 of 1 operations, client version 13 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 ready 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 validated 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: success 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 candidate ready 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - candidate rootPath: / 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - messages: [] 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: preCommit - successfulFromPrevious: [] 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 prepared 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 03:03:05,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 submitted to persistence 03:03:05,667 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 03:03:05,668 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 03:03:05,668 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:03:05,668 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 03:03:05,668 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 03:03:05,668 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 03:03:05,668 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 03:03:05,668 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:05,669 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: commit - successfulFromPrevious: [] 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from COMMIT_PENDING to COMMITTED 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:03:05,669 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$k] 03:03:05,670 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 03:03:05,670 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 03:03:05,670 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 03:03:05,670 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,670 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 76.97 μs 03:03:05,671 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 successfully completed direct commit 03:03:05,671 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 modifications completed with CommitTransactionReply [version=13] 03:03:05,671 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 03:03:05,671 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 03:03:05,671 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 03:03:05,671 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 03:03:05,671 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 03:03:05,671 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 03:03:05,671 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,671 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,671 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 03:03:05,671 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,671 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Queuing TransactionOperation 03:03:05,672 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:03:05,673 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 03:03:05,673 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 03:03:05,673 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,673 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 66.80 μs 03:03:05,673 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 03:03:05,673 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,673 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5568b025 03:03:05,674 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#985793000] 03:03:05,674 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#985793000] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:05,674 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#985793000, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0, version=13] 03:03:05,674 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:05,674 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#985793000)] 03:03:05,676 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 ReadData operation succeeded 03:03:05,677 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 closeTransaction called 03:03:05,677 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_27#985793000)] 03:03:05,677 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,677 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,677 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,677 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 03:03:05,677 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}] 03:03:05,677 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying 1 components for commit 03:03:05,677 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying transaction for shard cars 03:03:05,677 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 directCommit called 03:03:05,677 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 sending 1 batched modifications, ready: true 03:03:05,677 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,677 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 canCommit 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5568b025 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 of 1 operations, client version 13 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 ready 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 validated 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: success 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 candidate ready 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - candidate rootPath: / 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - messages: [] 03:03:05,679 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: preCommit - successfulFromPrevious: [] 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 prepared 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 03:03:05,680 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 submitted to persistence 03:03:05,681 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 03:03:05,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 03:03:05,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:03:05,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 03:03:05,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 03:03:05,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 03:03:05,681 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 03:03:05,682 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:05,682 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 03:03:05,683 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 7, nextIndex: 8 03:03:05,683 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 03:03:05,683 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 03:03:05,683 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 03:03:05,683 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 03:03:05,684 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,684 PM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: commit - successfulFromPrevious: [] 03:03:05,684 PM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from COMMIT_PENDING to COMMITTED 03:03:05,684 PM [cluster-test-shard-dispatcher-17] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:03:05,684 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$n] 03:03:05,684 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 03:03:05,684 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 03:03:05,684 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 03:03:05,684 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,685 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 135.2 μs 03:03:05,686 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 successfully completed direct commit 03:03:05,686 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 modifications completed with CommitTransactionReply [version=13] 03:03:05,686 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 03:03:05,686 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 7 03:03:05,686 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:05,686 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:05,686 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 03:03:05,686 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 03:03:05,686 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:05,686 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 03:03:05,686 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 03:03:05,686 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 03:03:05,686 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Queuing TransactionOperation 03:03:05,686 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:03:05,687 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 03:03:05,687 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 03:03:05,687 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:03:05,688 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 65.39 μs 03:03:05,691 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 03:03:05,691 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:05,691 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6dd2415 03:03:05,692 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#2022915654] 03:03:05,692 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#2022915654] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:05,693 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#2022915654, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0, version=13] 03:03:05,693 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:05,693 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#2022915654)] 03:03:05,698 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 ReadData operation succeeded 03:03:05,698 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 closeTransaction called 03:03:05,698 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_28#2022915654)] 03:03:05,899 PM [cluster-test-shard-dispatcher-17] [WARN] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 207, lastApplied : 7, commitIndex : 7 03:03:06,200 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#-1023616608] 03:03:06,200 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 03:03:06,201 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1377656660] 03:03:06,201 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1377656660] 03:03:06,202 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 03:03:06,202 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:06,202 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:06,202 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 03:03:06,202 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 03:03:06,202 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Queuing TransactionOperation 03:03:06,204 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 03:03:06,204 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 03:03:06,204 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6dd2415 03:03:06,204 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#12852288] 03:03:06,205 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#12852288, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0, version=13] 03:03:06,205 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:03:06,205 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#12852288)] 03:03:06,213 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 ReadData operation succeeded 03:03:06,213 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 closeTransaction called 03:03:06,213 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_29#12852288)] 03:03:06,213 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:03:06,214 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:06,214 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 03:03:06,214 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 03:03:06,214 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 03:03:06,214 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 03:03:06,215 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:06,215 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@38ab3d3 03:03:06,215 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:06,215 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@38ab3d3 03:03:06,215 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 03:03:06,215 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 03:03:06,216 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession] 03:03:06,216 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 03:03:06,216 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:06,216 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:06,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, matchIndex: 7, lastIndex: 7, isVoting: true 03:03:06,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Follower's log matches - sending ElectionTimeout 03:03:06,216 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 03:03:06,217 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leader transfer complete 03:03:06,466 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:03:06,626 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 03:03:06,626 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1377656660]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:06,626 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:06,626 PM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-236: Closing 03:03:06,626 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 03:03:06,626 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 03:03:06,627 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 03:03:06,627 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:03:06,627 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 03:03:06,628 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 03:03:06,628 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:06,628 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:06,634 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:06,634 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:06,634 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:06,634 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:06,636 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:06,647 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 03:03:06,648 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:03:06,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 03:03:06,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 03:03:06,648 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 03:03:06,648 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:06,649 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:07,641 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4501509045548228290] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:07,661 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-595773636353726061] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:07,671 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:07.668] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-200)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:07.673] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:07,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:07,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:03:07.694] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:07,719 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:07,728 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3084666751136499747] 03:03:07,728 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:07,730 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:07,730 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:07,738 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:07,738 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:07,746 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:07,746 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:07,746 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:07,753 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:07,764 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5081737379378624844] 03:03:07,765 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:07,777 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:07,777 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:07,777 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:07,777 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:07,783 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:07,795 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8953930469968284424] 03:03:07,795 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:07,807 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:07,807 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:07,808 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:07,808 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:07,810 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:07,811 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:07,817 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 03:03:07,844 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 03:03:07,844 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:07,844 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3b8ba24 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@d7f4423 03:03:07,844 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@940d154 03:03:07,844 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709823787844, None)) 03:03:07,845 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:07,845 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#381179470] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:07,847 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:07,847 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823787843,0,0)) 03:03:07,848 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@d7f4423 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7dc4a70c 03:03:07,848 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 03:03:07,848 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:07,850 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:07,852 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@d587c58 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,855 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] 03:03:07,861 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:07,861 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:07,865 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:07,865 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1495987981] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:07,866 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 03:03:07,866 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-857167109 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,867 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#796543056] 03:03:07,867 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 03:03:07,867 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-857167109] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:07,867 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#796543056] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:07,868 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:07,873 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:07,873 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,873 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:07,873 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,874 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:07,883 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 03:03:07,883 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:07,883 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@70bc3915 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e7bd972 03:03:07,883 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7a445265 03:03:07,887 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:07,887 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:07,887 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:03:07,887 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:03:07,887 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 03:03:07,888 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 03:03:07,888 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 03:03:07,888 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 03:03:07,888 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709823787887, None)) 03:03:07,888 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 03:03:07,888 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823787886,0,0)) 03:03:07,888 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e7bd972 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7e30bd14 03:03:07,890 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@63066363 03:03:07,892 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 03:03:07,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:07,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,893 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,893 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,893 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#-1309838591] 03:03:07,894 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 03:03:07,894 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 03:03:07,904 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#-1309838591] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:07,904 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1929555608 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,904 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#1929555608] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:07,904 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#955393632] 03:03:07,905 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#955393632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:07,909 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:07,910 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:07,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 03:03:07,910 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 03:03:07,910 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:03:07,910 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:03:07,910 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:07,910 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 03:03:07,910 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 03:03:07,910 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 03:03:07,943 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#381179470] 03:03:07,943 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 03:03:07,943 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:03:08,169 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 03:03:08,169 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 03:03:08,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 03:03:08,399 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,399 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 03:03:08,435 PM [cluster-test-shard-dispatcher-23] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:08,719 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 03:03:08,719 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,719 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 03:03:08,750 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:08,751 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,751 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:08,751 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,751 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,751 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,751 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:08,751 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:08,758 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:08,758 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,758 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,758 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:08,758 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,758 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,758 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,822 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:08,959 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 03:03:08,959 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,959 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 03:03:08,959 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,959 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,961 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:08,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:08,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 03:03:08,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:08,962 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:08,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 03:03:08,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:08,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:08,963 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:08,963 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:08,964 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:03:08,964 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:08,964 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:08,964 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:08,964 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 03:03:08,964 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 03:03:08,965 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 03:03:08,965 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:08,965 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 03:03:08,965 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:08,965 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:08,965 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 03:03:08,965 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 03:03:08,965 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:08,966 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 03:03:08,966 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:08,967 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 03:03:08,967 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 03:03:08,967 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:08,967 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:08,967 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 03:03:08,967 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 03:03:09,1 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testRaftCallbackDuringLeadershipDrop commit rateLimit adjusted to 85.79685415538616 03:03:09,1 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,1 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#381179470] 03:03:09,1 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:09,1 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 03:03:09,1 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying 1 components for commit 03:03:09,1 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying transaction for shard cars 03:03:09,1 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@76a6b700] 03:03:09,1 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 03:03:09,1 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 canCommit 03:03:09,1 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@76a6b700] 03:03:09,1 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672)] for shard cars 03:03:09,1 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:03:09,1 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6582ff2e 03:03:09,2 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 = [ 1 ] 03:03:09,2 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672)] 03:03:09,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, disabling transaction tracking 03:03:09,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 03:03:09,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 03:03:09,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:09,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:09,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:09,2 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 validated 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: success 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 candidate ready 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - messages: [] 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 prepared 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 03:03:09,3 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 03:03:09,4 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:09,4 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:09,4 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:09,4 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,4 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 submitted to persistence 03:03:09,5 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 03:03:09,5 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 03:03:09,5 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 03:03:09,5 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 03:03:09,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 03:03:09,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:09,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 03:03:09,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 03:03:09,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 03:03:09,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:09,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 03:03:09,12 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=10403739} is already disabled 03:03:09,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:09,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 03:03:09,13 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 03:03:09,13 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 03:03:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 03:03:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 03:03:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 03:03:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 03:03:09,14 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 03:03:09,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 03:03:09,15 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 03:03:09,15 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%23252015672$a] 03:03:09,16 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 successfully completed direct commit 03:03:09,16 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed successfully 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1522334476] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 03:03:09,16 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:09,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 03:03:09,17 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1522334476] 03:03:09,17 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:03:09,17 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 129.7 μs 03:03:09,17 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1522334476] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,18 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:09,21 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 03:03:09,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 03:03:09,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:09,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 03:03:09,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 03:03:09,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 03:03:09,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#734914830] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 03:03:09,25 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 03:03:09,25 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#734914830] 03:03:09,26 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:03:09,26 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#734914830] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,26 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 97.53 μs 03:03:09,27 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#371932255] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,27 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:09,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 03:03:09,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:09,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3f86f14 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@46fae10f 03:03:09,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4f50ee90 03:03:09,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709823789029, None)) 03:03:09,29 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:09,29 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,29 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823789028,0,0)) 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:09,30 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@46fae10f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@530fb6d8 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2c3ca70f 03:03:09,30 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 03:03:09,30 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#371932255] 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:09,30 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#572378600] 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#572378600] 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,31 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#572378600] 03:03:09,31 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 03:03:09,31 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1495987981] 03:03:09,31 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#572378600] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,31 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,31 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1069575583 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,31 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:09,31 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6582ff2e 03:03:09,32 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1069575583] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,32 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 03:03:09,32 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#1262713913] 03:03:09,32 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 03:03:09,32 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#1262713913] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:09,33 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:09,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 03:03:09,33 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 03:03:09,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:03:09,33 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:03:09,33 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 03:03:09,33 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,33 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,33 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 03:03:09,33 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 03:03:09,34 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#381179470] 03:03:09,34 PM [pool-4-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying 1 components for commit 03:03:09,34 PM [pool-4-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying transaction for shard cars 03:03:09,34 PM [pool-4-thread-1] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 = [ 1 ] 03:03:09,34 PM [pool-4-thread-1] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672)] 03:03:09,34 PM [pool-4-thread-1] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 canCommit 03:03:09,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 validated 03:03:09,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:09,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: success 03:03:09,35 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 candidate ready 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - candidate rootPath: / 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - messages: [] 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 prepared 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 71 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]]] 03:03:09,36 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 submitted to persistence 03:03:09,41 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 03:03:09,137 PM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] 03:03:09,139 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672] 03:03:09,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:09,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 03:03:09,139 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-241: Closing 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 03:03:09,140 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:09,140 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 03:03:09,388 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:09,388 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:03:09,389 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 03:03:09,389 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 03:03:09,390 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:09,391 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 03:03:09,391 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 03:03:09,391 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 0 - updating term 03:03:09,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:09,391 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 03:03:09,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 03:03:09,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 03:03:09,391 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 03:03:09,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 03:03:09,391 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:09,391 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 03:03:09,391 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 03:03:09,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,391 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 03:03:09,397 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:03:09,397 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 03:03:09,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:03:09,397 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:09,397 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:09,397 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 03:03:09,399 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:03:09,399 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 03:03:09,399 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:09,399 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 03:03:09,399 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,399 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 03:03:09,399 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:09,400 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 03:03:09,400 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:09,400 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:09,400 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 03:03:09,400 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 03:03:09,400 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 03:03:09,400 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 03:03:09,400 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:03:09,400 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,400 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 03:03:09,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 03:03:09,402 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:09,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 03:03:09,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:03:09,403 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 03:03:09,403 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 03:03:09,403 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 03:03:09,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:03:09,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 03:03:09,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:09,403 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 03:03:09,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 03:03:09,403 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 03:03:09,403 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 03:03:09,403 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, size=71} 03:03:09,403 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: commit - successfulFromPrevious: [] 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%23252015672$b] 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]] - lastApplied: 3, lastIndex: 3 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,404 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 successfully completed direct commit 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:03:09,404 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:03:09,404 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed successfully 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:03:09,404 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-244: Closing 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 03:03:09,404 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 03:03:09,404 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 29.75 μs 03:03:09,404 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 03:03:09,405 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#2084557683] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 03:03:09,405 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#2084557683] 03:03:09,405 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:03:09,405 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#2084557683] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,405 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 112.8 μs 03:03:09,405 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 03:03:09,405 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 03:03:09,405 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:09,405 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 03:03:09,405 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 03:03:09,406 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 03:03:09,406 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 03:03:09,406 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 03:03:09,406 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:03:09,406 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] is present: existingEntryTerm: 4 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] is present: existingEntryTerm: 4 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] is present: existingEntryTerm: 4 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] is present: existingEntryTerm: 6 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 4 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 03:03:09,406 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 03:03:09,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:03:09,407 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 03:03:09,407 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 03:03:09,407 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:03:09,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 03:03:09,407 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 03:03:09,407 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:03:09,407 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 18.96 μs 03:03:09,409 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:03:09,409 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 33.95 μs 03:03:09,448 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:09,449 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,449 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 03:03:09,449 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 03:03:09,449 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 03:03:09,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 03:03:09,449 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 03:03:09,456 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:09,456 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:03:09,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 03:03:09,456 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:03:09,456 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:09,457 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#371932255] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#371932255] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,457 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,457 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 03:03:09,457 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 03:03:09,457 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 03:03:09,457 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 03:03:09,457 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:09,457 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@673063ff 03:03:09,457 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:09,457 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@673063ff 03:03:09,457 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 03:03:09,458 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:09,458 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:09,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:03:09,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 03:03:09,459 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 03:03:09,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:09,459 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:09,460 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, matchIndex: 3, lastIndex: 3, isVoting: true 03:03:09,460 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Follower's log matches - sending ElectionTimeout 03:03:09,460 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:09,460 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:09,460 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leader transfer complete 03:03:09,769 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:03:09,869 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 03:03:09,869 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#252015672]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:09,870 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,870 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-245: Closing 03:03:09,870 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 03:03:09,870 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:09,870 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:03:09,870 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:03:09,871 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 03:03:09,871 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:03:09,871 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:03:09,871 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#381179470] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#381179470] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:09,871 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:09,871 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:09,877 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:09,877 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:09,877 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:09,877 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:09,879 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:09,890 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:03:09,890 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:03:09,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 03:03:09,890 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 03:03:09,890 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:03:09,891 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:09,891 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:10,891 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3084666751136499747] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:10,907 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:10,907 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:10,910 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:10.911] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:10,912 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5081737379378624844] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:10,920 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:10,920 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:10,922 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:10.923] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:10,948 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:10,960 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5248121769931501527] 03:03:10,960 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:10,962 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:10,962 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:10,962 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:10,962 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:10,977 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:10,977 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:10,977 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:10,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:10,987 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3981770712293298494] 03:03:10,987 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:10,989 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:10,989 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:10,995 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:10,995 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:11,7 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:11,23 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4937225596331835693] 03:03:11,23 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:11,23 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:11,23 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:11,24 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:11,24 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:11,28 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:11,29 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:11,31 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 03:03:11,45 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 03:03:11,46 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-372763267] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:11,47 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:11,50 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 03:03:11,50 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,50 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:11,50 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,51 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:11,57 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 03:03:11,57 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:03:11,57 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@71913206 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7b9d9659 03:03:11,57 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@97bc40e 03:03:11,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1709823791057, None)) 03:03:11,58 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823791056,0,0)) 03:03:11,58 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@62b004c3 03:03:11,58 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7b9d9659 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@553c74ab 03:03:11,59 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 03:03:11,59 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:11,59 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,60 PM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,60 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,60 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-327571046] 03:03:11,61 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 03:03:11,61 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:11,61 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-327571046] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:11,62 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#951030749 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,62 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 03:03:11,62 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#951030749] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:11,63 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:11,65 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-1410549544] 03:03:11,65 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-1410549544] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:11,65 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 03:03:11,70 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:11,70 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:11,70 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:11,71 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:11,71 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 03:03:11,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 03:03:11,71 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 03:03:11,71 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:11,72 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-651516000] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7bebf553 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@752c045e 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@71139e23 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 03:03:11,74 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1709823791074, None)) 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823791073,0,0)) 03:03:11,74 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@752c045e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5efdc7d 03:03:11,74 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 03:03:11,76 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 03:03:11,76 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 03:03:11,76 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 03:03:11,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@74bd5254 03:03:11,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:11,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 03:03:11,77 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:11,77 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,77 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:03:11,78 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 03:03:11,78 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#980887008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:11,78 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 03:03:11,79 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1071418748] 03:03:11,79 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-1529879536 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 03:03:11,79 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-1529879536] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:11,79 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1071418748] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:11,80 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:11,80 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:11,80 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 03:03:11,80 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 03:03:11,80 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:11,80 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 03:03:11,80 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,80 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 03:03:11,80 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 03:03:11,80 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 03:03:11,125 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-372763267] 03:03:11,125 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 03:03:11,125 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-327571046] 03:03:11,306 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:11,306 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 03:03:11,306 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 03:03:11,307 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,307 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 03:03:11,307 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:11,307 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 03:03:11,307 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:11,307 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 03:03:11,307 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 03:03:11,307 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 03:03:11,606 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 03:03:11,606 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,607 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 03:03:11,684 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:11,876 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 03:03:11,876 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,877 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 03:03:11,978 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:11,978 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 03:03:11,978 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,978 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:11,978 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:03:11,978 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:03:11,978 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:03:11,978 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:11,978 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:11,984 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:11,984 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 03:03:11,984 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:03:11,984 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:11,985 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:03:11,985 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:03:12,41 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:12,146 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 03:03:12,146 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:03:12,146 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 4 03:03:12,146 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 03:03:12,148 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:12,148 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 03:03:12,148 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:12,148 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:12,148 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=null 03:03:12,148 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:12,148 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:03:12,149 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 03:03:12,150 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 03:03:12,150 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 03:03:12,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 03:03:12,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:03:12,151 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 03:03:12,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:12,151 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:12,151 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 03:03:12,151 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 03:03:12,152 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 03:03:12,152 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:12,183 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionChainWithSingleShard commit rateLimit adjusted to 85.79685415538616 03:03:12,183 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,183 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-651516000] 03:03:12,183 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:03:12,183 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 03:03:12,183 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 03:03:12,183 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:03:12,184 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 03:03:12,184 PM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,184 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 03:03:12,184 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:12,184 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 03:03:12,184 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 03:03:12,184 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 03:03:12,184 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - creating WRITE_ONLY transaction context 03:03:12,184 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:12,184 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 03:03:12,184 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 03:03:12,184 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 03:03:12,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, disabling transaction tracking 03:03:12,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 03:03:12,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 03:03:12,186 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@79cc0f46 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 of 1 operations, client version 13 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 ready 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 03:03:12,187 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 validated 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: success 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 candidate ready 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - messages: [] 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 prepared 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 03:03:12,188 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 submitted to persistence 03:03:12,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 03:03:12,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 03:03:12,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:12,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 03:03:12,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 03:03:12,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 1 03:03:12,189 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:12,190 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 03:03:12,190 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 03:03:12,190 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 03:03:12,190 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 03:03:12,190 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 0, nextIndex: 1 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=4460821} is already disabled 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}]]] 03:03:12,191 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}]]] 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=173} 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 0 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:03:12,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 2, nextIndex: 3 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} has no callback 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard$a] 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-96214339] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 03:03:12,193 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-96214339] 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:12,193 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 03:03:12,193 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 03:03:12,194 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-96214339] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:12,194 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 170.2 μs 03:03:12,195 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - previous Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 readied - proceeding to FindPrimaryShard 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 03:03:12,195 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=13] 03:03:12,195 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 03:03:12,195 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 03:03:12,195 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 2 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 03:03:12,195 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:03:12,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-23277323] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 03:03:12,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 03:03:12,196 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-23277323] 03:03:12,196 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 03:03:12,196 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 03:03:12,196 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 124.4 μs 03:03:12,196 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#-23277323] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:12,201 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 03:03:12,201 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 03:03:12,201 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7d1256fe 03:03:12,201 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#-1593095978] 03:03:12,202 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#-1593095978] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:12,203 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#-1593095978, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0, version=13] 03:03:12,203 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:12,203 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#-1593095978)] 03:03:12,205 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 ReadData operation succeeded 03:03:12,205 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,205 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:12,205 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:12,205 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 03:03:12,205 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 03:03:12,205 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 03:03:12,206 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,206 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 03:03:12,206 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7d1256fe 03:03:12,207 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#1307559532] 03:03:12,207 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#1307559532] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:12,208 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#1307559532, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, version=13] 03:03:12,208 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:12,208 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#1307559532)] 03:03:12,211 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 03:03:12,211 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:03:12,211 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:03:12,211 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:03:12,211 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: false 03:03:12,211 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#1307559532)] 03:03:12,211 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#1307559532)] 03:03:12,214 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with BatchedModificationsReply [numBatched=2] 03:03:12,214 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 DataExists operation succeeded 03:03:12,214 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:03:12,214 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#1307559532)] 03:03:12,225 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 03:03:12,225 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 03:03:12,225 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 03:03:12,225 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 03:03:12,225 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 successfully completed direct commit 03:03:12,225 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying 1 components for commit 03:03:12,225 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying transaction for shard cars 03:03:12,225 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 directCommit called 03:03:12,225 PM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: true 03:03:12,225 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_31#1307559532)] 03:03:12,225 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 canCommit 03:03:12,227 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, client version 13 03:03:12,227 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ready 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 validated 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: success 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 candidate ready 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - messages: [] 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 prepared 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 03:03:12,228 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 03:03:12,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:12,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:03:12,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 03:03:12,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}]]] 03:03:12,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 submitted to persistence 03:03:12,229 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 03:03:12,230 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}]]] 03:03:12,230 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 03:03:12,230 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:03:12,230 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=229} 03:03:12,230 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 03:03:12,230 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 03:03:12,230 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:12,230 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard_shard-cars-member-2%3Adatastore-testTransactionChainWithSingleShard%400%3A1-2_31%231307559532$i] 03:03:12,231 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 03:03:12,231 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 03:03:12,232 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 116.8 μs 03:03:12,232 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:12,232 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 03:03:12,232 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 03:03:12,232 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 03:03:12,232 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 03:03:12,232 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=13] 03:03:12,232 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 03:03:12,232 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 03:03:12,232 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 successfully completed direct commit 03:03:12,233 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:03:12,233 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 03:03:12,233 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 03:03:12,233 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 03:03:12,233 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 96.47 μs 03:03:12,233 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 03:03:12,234 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:12,234 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:12,234 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 03:03:12,234 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 03:03:12,234 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 03:03:12,234 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 03:03:12,234 PM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 03:03:12,235 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1315579908] 03:03:12,235 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1315579908] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:12,235 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1315579908, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0, version=13] 03:03:12,235 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:03:12,236 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1315579908)] 03:03:12,237 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 03:03:12,238 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:03:12,238 PM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 03:03:12,238 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 03:03:12,238 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 03:03:12,238 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 03:03:12,238 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:03:12,238 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:12,238 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:12,238 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c3dcbc7 03:03:12,238 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4c3dcbc7 03:03:12,238 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 03:03:12,238 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:12,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard] 03:03:12,239 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:03:12,239 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:12,239 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:12,241 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithSingleShard, matchIndex: 3, lastIndex: 3, isVoting: true 03:03:12,241 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 03:03:12,241 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:12,241 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leader transfer complete 03:03:12,656 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 03:03:12,656 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testTransactionChainWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-327571046]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:12,657 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 03:03:12,657 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-250: Closing 03:03:12,657 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 03:03:12,657 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 03:03:12,657 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 03:03:12,657 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:03:12,658 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 03:03:12,658 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 03:03:12,658 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 03:03:12,658 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-372763267] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-372763267] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:12,658 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-372763267] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-372763267] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:12,659 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:12,659 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:12,665 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:12,665 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:12,665 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:12,665 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:12,666 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:12,677 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 03:03:12,678 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 03:03:12,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 03:03:12,678 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 03:03:12,678 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 03:03:12,678 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:12,678 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:13,680 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5248121769931501527] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:13,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3981770712293298494] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:13,701 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:13,701 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:13,703 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:13.705] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:13,714 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:13,714 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:13,716 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:13.717] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:13,742 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:13,750 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-150718383177043823] 03:03:13,751 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:13,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:13,756 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:13,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:13,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:13,772 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:13,773 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:13,773 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:13,773 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:13,777 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2960169565192808081] 03:03:13,777 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:13,779 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:13,779 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:13,792 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:13,792 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:13,806 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:13,821 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:13,823 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:13,825 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7220636571757604592] 03:03:13,825 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:13,826 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:13,827 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:13,832 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:13,832 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:13,838 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:13,838 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:13,854 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:13,854 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:13,858 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:13,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:13,868 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:14,883 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:14.884] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:14,891 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:14,891 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:03:14.895] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:14,920 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:14,927 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2550102573070384911] 03:03:14,927 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:14,928 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:14,928 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:14,930 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:14,930 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:14,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:14,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:14,941 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:14,943 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:14,949 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3435269498837774290] 03:03:14,949 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:14,951 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:14,951 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:14,958 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:14,959 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:14,967 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:14,979 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1970714044616895372] 03:03:14,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:14,981 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:14,981 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:14,987 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:14,987 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:14,994 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:14,996 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:14,998 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 03:03:15,16 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 closeTransaction called 03:03:15,16 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_32#1315579908)] 03:03:15,16 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 closeTransaction called 03:03:15,16 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_30#-1593095978)] 03:03:15,31 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:15,36 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:15,36 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 03:03:15,36 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:03:15,36 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@34e616b0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a21ee5c 03:03:15,37 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 03:03:15,39 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-674943393] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:15,41 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:15,43 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:15,46 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 03:03:15,46 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,46 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:15,46 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,51 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3e1e7dc0 03:03:15,51 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1709823795036, None)) 03:03:15,51 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823795035,0,0)) 03:03:15,51 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a21ee5c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4aad7c99 03:03:15,51 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@41a0e789 03:03:15,52 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 03:03:15,52 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:15,52 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,52 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:15,53 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,53 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,53 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189] 03:03:15,53 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 03:03:15,54 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:15,54 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 03:03:15,56 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:15,56 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-788213372 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,56 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#471815357] 03:03:15,56 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-788213372] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:15,56 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 03:03:15,56 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#471815357] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:15,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:15,58 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:15,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:15,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 03:03:15,58 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 03:03:15,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 03:03:15,58 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 03:03:15,58 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 03:03:15,58 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 03:03:15,63 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#893977818] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:15,63 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 03:03:15,66 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 03:03:15,66 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,66 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:15,66 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,68 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 03:03:15,68 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:03:15,68 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5df94177 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e69bfba 03:03:15,68 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1dc23104 03:03:15,70 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1709823795070, None)) 03:03:15,70 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823795069,0,0)) 03:03:15,70 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3a842533 03:03:15,70 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e69bfba to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1c86e8d7 03:03:15,71 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 03:03:15,71 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:15,71 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,71 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,71 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,71 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1676828314] 03:03:15,72 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 03:03:15,72 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1676828314] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:15,72 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#-675959160] 03:03:15,72 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-2055527853 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,72 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-2055527853] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:15,72 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 03:03:15,73 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#-675959160] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:15,73 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:15,73 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:15,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 03:03:15,73 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 03:03:15,73 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:15,73 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 03:03:15,73 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 03:03:15,73 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,74 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 03:03:15,74 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 03:03:15,121 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-674943393] 03:03:15,122 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 03:03:15,122 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189] 03:03:15,288 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:15,288 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 03:03:15,288 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 03:03:15,288 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,288 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 03:03:15,288 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:15,288 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 03:03:15,289 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:15,289 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 03:03:15,289 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 03:03:15,289 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 03:03:15,607 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:03:15,607 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,607 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 03:03:15,690 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:15,837 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:03:15,838 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,838 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 03:03:15,948 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:15,948 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:15,949 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:15,949 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,949 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,949 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:15,949 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,949 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,949 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,955 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:15,955 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,955 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,955 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:15,955 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,955 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:15,995 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:16,117 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:03:16,117 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,117 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 03:03:16,117 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,119 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:16,119 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 03:03:16,119 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:16,119 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:16,119 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 03:03:16,119 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:16,119 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,119 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:16,120 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:16,120 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 03:03:16,121 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 03:03:16,121 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 03:03:16,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 03:03:16,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,122 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 03:03:16,122 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 03:03:16,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:16,122 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:16,122 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 03:03:16,123 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 03:03:16,123 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:16,181 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#893977818] 03:03:16,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 03:03:16,181 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1676828314] 03:03:16,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,181 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#893977818] 03:03:16,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 03:03:16,181 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1676828314] 03:03:16,194 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,194 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,194 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 03:03:16,196 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 03:03:16,196 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:16,196 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:16,196 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-127-0 of 3 operations, client version 13 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-127-0 ready 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 validated 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-127-0: success 03:03:16,197 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 candidate ready 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-127-0: canCommit - candidate rootPath: / 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-127-0: canCommit - messages: [] 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-127-0: preCommit - successfulFromPrevious: [] 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 prepared 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-127-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,198 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 submitted to persistence 03:03:16,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 03:03:16,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:03:16,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:16,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:03:16,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:16,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 03:03:16,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:16,199 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:16,199 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:16,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:16,200 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 03:03:16,200 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 03:03:16,200 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:16,200 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 03:03:16,200 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 03:03:16,200 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:03:16,200 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 03:03:16,200 PM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 03:03:16,200 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=4155463} is already disabled 03:03:16,201 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:16,201 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,201 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 03:03:16,201 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:16,202 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-127-0: commit - successfulFromPrevious: [] 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 changing state from COMMIT_PENDING to COMMITTED 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-212#2032959546] 03:03:16,203 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-906818613] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 03:03:16,203 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-906818613] 03:03:16,203 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 03:03:16,204 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 03:03:16,204 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:16,204 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 03:03:16,204 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 130.3 μs 03:03:16,204 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-906818613] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:16,205 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 03:03:16,205 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-674943393] 03:03:16,205 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:16,205 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 03:03:16,205 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@ea2856e] 03:03:16,205 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@ea2856e] 03:03:16,205 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189)] for shard cars 03:03:16,205 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:03:16,205 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75ff13f2 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-127-0 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1456118224] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,206 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 03:03:16,206 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1456118224] 03:03:16,207 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 03:03:16,207 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 83.20 μs 03:03:16,207 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#-1456118224] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:16,208 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,208 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-128-0 of 1 operations, client version 13 03:03:16,208 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,208 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-128-0 ready 03:03:16,209 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0 canCommit 03:03:16,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189)] resolved 03:03:16,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: successfully resolved all cohort actors 03:03:16,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0 finishCanCommit 03:03:16,210 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-128-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189)] 03:03:16,210 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189)] 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 validated 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-128-0: success 03:03:16,210 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:03:16,210 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: canCommit returning result: true 03:03:16,210 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0 commit 03:03:16,210 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0 finish commit 03:03:16,210 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-128-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189)] 03:03:16,210 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189)] 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 candidate ready 03:03:16,210 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-128-0: canCommit - candidate rootPath: / 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-128-0: canCommit - messages: [] 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-128-0: preCommit - successfulFromPrevious: [] 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 prepared 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-128-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 03:03:16,211 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 submitted to persistence 03:03:16,212 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:16,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 03:03:16,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:03:16,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:16,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 03:03:16,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:16,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 03:03:16,212 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-128-0: commit - successfulFromPrevious: [] 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 changing state from COMMIT_PENDING to COMMITTED 03:03:16,213 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23-150950189$a] 03:03:16,213 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: commit succeeded 03:03:16,213 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 03:03:16,214 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 03:03:16,214 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 73.02 μs 03:03:16,214 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 03:03:16,214 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:16,214 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 03:03:16,214 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:16,214 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@75ff13f2 03:03:16,214 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 03:03:16,214 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:03:16,214 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,214 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 03:03:16,214 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 03:03:16,214 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 03:03:16,215 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 03:03:16,215 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:16,215 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:16,215 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7dc2b401 03:03:16,215 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7dc2b401 03:03:16,215 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 03:03:16,215 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:16,217 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:16,217 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 2 03:03:16,217 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:03:16,217 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 2 03:03:16,217 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 03:03:16,217 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 03:03:16,217 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-128-0 03:03:16,218 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 03:03:16,218 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 03:03:16,218 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader] 03:03:16,218 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 03:03:16,218 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 03:03:16,218 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:16,218 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:16,218 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 111.7 μs 03:03:16,219 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 03:03:16,219 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 03:03:16,219 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:16,219 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leader transfer complete 03:03:16,627 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 03:03:16,627 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-150950189]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:16,628 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,628 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-255: Closing 03:03:16,628 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 03:03:16,628 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 03:03:16,629 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 03:03:16,629 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:03:16,630 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 03:03:16,630 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 03:03:16,630 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 03:03:16,633 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:16,633 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:16,639 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:16,639 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:16,639 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:16,639 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:16,641 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:16,648 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 03:03:16,649 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:03:16,649 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 03:03:16,649 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 03:03:16,649 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 03:03:16,650 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:16,650 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:17,654 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2550102573070384911] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:17,664 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3435269498837774290] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:17,673 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:17,673 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:17,674 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:17.676] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:17,682 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:17,682 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:17,685 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:17.686] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:17,705 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:17,714 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3188992780518791367] 03:03:17,714 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:17,717 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:17,717 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:17,718 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:17,718 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:17,737 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:17,737 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:17,737 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:17,739 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:17,747 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8122587449525440435] 03:03:17,747 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:17,749 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:17,749 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:17,761 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:17,761 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:17,772 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:17,783 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1104376736450241282] 03:03:17,783 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:17,786 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:17,786 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:17,790 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:17,790 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:17,797 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:17,798 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:17,800 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 03:03:17,814 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:17,819 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 03:03:17,820 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 03:03:17,820 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,820 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:17,820 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,823 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:17,823 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 03:03:17,823 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:03:17,824 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@22712ea to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2020189b 03:03:17,824 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@336f50de 03:03:17,824 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1709823797824, None)) 03:03:17,824 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823797823,0,0)) 03:03:17,824 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2020189b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6208eda5 03:03:17,824 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 03:03:17,825 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7a86aa4b 03:03:17,825 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:17,825 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,825 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:17,826 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,826 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,826 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#909702101] 03:03:17,827 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 03:03:17,827 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:17,827 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#909702101] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:17,827 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1002824262 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,827 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1002824262] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:17,827 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#1785776395] 03:03:17,828 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#1785776395] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:17,829 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 03:03:17,834 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:17,837 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 03:03:17,839 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-373841940] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:17,839 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:17,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:17,839 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:17,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 03:03:17,839 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 03:03:17,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:17,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 03:03:17,840 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 03:03:17,840 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 03:03:17,840 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 03:03:17,840 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 03:03:17,841 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 03:03:17,841 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,842 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:17,842 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,842 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 03:03:17,842 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:03:17,842 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@65e1e6ef to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@cc29520 03:03:17,842 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@69a6f9a7 03:03:17,842 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1709823797842, None)) 03:03:17,842 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:17,843 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823797841,0,0)) 03:03:17,843 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@cc29520 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@525bdd92 03:03:17,843 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@e1095a8 03:03:17,843 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 03:03:17,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:17,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,844 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,844 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-398819566] 03:03:17,845 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 03:03:17,845 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-398819566] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:17,845 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-1294698145] 03:03:17,845 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-112121030 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,845 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-112121030] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:17,845 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-1294698145] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:17,846 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 03:03:17,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:17,846 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:17,846 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 03:03:17,846 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 03:03:17,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:17,846 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 03:03:17,847 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:17,847 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 03:03:17,847 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 03:03:17,847 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 03:03:17,894 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] 03:03:17,894 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 03:03:17,894 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#909702101] 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:18,119 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 03:03:18,119 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 03:03:18,120 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 03:03:18,379 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 03:03:18,379 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,379 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 03:03:18,384 PM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:18,699 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 03:03:18,699 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,699 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 03:03:18,730 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:18,730 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:18,730 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:18,730 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,730 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,730 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:18,731 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,731 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,731 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,750 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:18,750 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,750 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,750 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:18,750 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,750 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,803 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:18,949 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 03:03:18,949 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,949 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 03:03:18,949 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:18,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 03:03:18,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:18,951 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:18,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 03:03:18,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:18,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:18,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:18,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:18,953 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 03:03:18,954 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 03:03:18,954 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 03:03:18,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 03:03:18,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,955 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 03:03:18,955 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 03:03:18,955 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:18,955 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:18,955 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 03:03:18,957 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 03:03:18,957 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:18,957 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 192853135, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@1015db31} 03:03:18,958 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithSingleShard commit rateLimit adjusted to 71.23365547131583 03:03:18,958 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 03:03:18,958 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-373841940] 03:03:18,958 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:18,958 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:03:18,958 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 03:03:18,959 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 03:03:18,959 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 03:03:18,959 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 03:03:18,959 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 03:03:18,959 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 03:03:18,959 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 03:03:18,960 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] for shard cars 03:03:18,960 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] found - creating WRITE_ONLY transaction context 03:03:18,960 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:18,960 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 03:03:18,960 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 03:03:18,960 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, disabling transaction tracking 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 03:03:18,961 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 03:03:18,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:03:18,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} 03:03:18,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1a6bba9d 03:03:18,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 03:03:18,962 PM [cluster-test-shard-dispatcher-12] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:03:18,962 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 03:03:18,963 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 03:03:18,964 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 03:03:18,964 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 03:03:18,964 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:18,964 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 03:03:18,964 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 03:03:18,964 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 1 03:03:18,964 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:18,965 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:03:18,965 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:03:18,966 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@3d7712f6 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.SingleCommitCohortProxy$1.onComplete(SingleCommitCohortProxy.java:60) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:745) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:768) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:373) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:03:18,965 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 03:03:18,967 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 03:03:18,966 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 0, nextIndex: 1 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=5745373} is already disabled 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,967 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 03:03:18,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 2 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 0 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 0 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:03:18,968 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 03:03:18,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 03:03:18,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 1, nextIndex: 2 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 1 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} has no callback 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:18,969 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 03:03:18,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 1 03:03:18,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:18,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 1 03:03:18,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 03:03:18,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73} 03:03:18,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:03:18,971 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 03:03:18,976 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@3d7712f6 is no longer running 03:03:18,976 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1f1b64c8 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1f1b64c8 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:18,977 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:18,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard] 03:03:18,979 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:03:18,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:18,979 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:18,980 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 03:03:18,980 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 03:03:18,980 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:18,980 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leader transfer complete 03:03:19,192 PM [cluster-test-shard-dispatcher-31] [WARN] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 210, lastApplied : 1, commitIndex : 1 03:03:19,389 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 03:03:19,389 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#909702101]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:19,390 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:19,390 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-260: Closing 03:03:19,390 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 03:03:19,390 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 03:03:19,390 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 03:03:19,391 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:03:19,391 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 03:03:19,391 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 03:03:19,391 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 03:03:19,391 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:19,391 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:19,391 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:19,391 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1093396719] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:19,392 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:19,392 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:19,398 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:19,398 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:19,398 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:19,398 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:19,401 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:19,412 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 03:03:19,412 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:03:19,413 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 03:03:19,413 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 03:03:19,413 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 03:03:19,413 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:19,413 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:20,413 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3188992780518791367] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:20,427 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:20,427 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:20,429 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:20.432] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:20,432 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8122587449525440435] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:20,449 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:20,449 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:03:20.453] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:20,471 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:20,483 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6859000036478932141] 03:03:20,483 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:20,489 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:20,489 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:20,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:20,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:20,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:20,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:20,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:20,501 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:20,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4611601372356322463] 03:03:20,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:20,518 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:20,518 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:20,525 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:20,525 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:20,555 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:03:20,570 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4403680964462184070] 03:03:20,570 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:20,584 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:20,586 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:20,588 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 03:03:20,590 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:20,590 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:20,590 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:20,590 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:20,606 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:20,609 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:20,618 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 03:03:20,620 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-816216791] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:20,622 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 03:03:20,622 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:03:20,622 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1f3a1a46 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a3897c5 03:03:20,622 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5cf51675 03:03:20,622 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1709823800622, None)) 03:03:20,623 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823800621,0,0)) 03:03:20,623 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a3897c5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@78765053 03:03:20,623 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 03:03:20,624 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 03:03:20,624 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,624 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:20,624 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:20,624 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,624 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6ce872d1 03:03:20,624 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:20,624 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,626 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:20,627 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 03:03:20,640 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,640 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,640 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1376310564] 03:03:20,640 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 03:03:20,640 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1376310564] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:20,641 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#531214940 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,641 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#531214940] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:20,641 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-1357139336] 03:03:20,641 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 03:03:20,641 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-1357139336] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:20,644 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:20,644 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:20,646 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:20,648 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:20,650 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 03:03:20,650 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 03:03:20,656 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 03:03:20,657 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1530589622] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:20,661 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 03:03:20,661 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 03:03:20,661 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 03:03:20,661 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 03:03:20,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 03:03:20,671 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 03:03:20,671 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:03:20,671 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1c2df6bc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7bb98b61 03:03:20,671 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4b920715 03:03:20,671 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 03:03:20,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,671 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:20,671 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,672 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1709823800671, None)) 03:03:20,672 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823800670,0,0)) 03:03:20,672 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7bb98b61 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@f7ab5f2 03:03:20,672 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@540ccd3e 03:03:20,673 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 03:03:20,673 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:20,673 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,674 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,674 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#148368335] 03:03:20,674 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 03:03:20,674 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#148368335] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:20,675 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#2095248475 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,675 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-653986615] 03:03:20,675 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#2095248475] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:20,677 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 03:03:20,678 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:20,678 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:20,678 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 03:03:20,678 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 03:03:20,678 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 03:03:20,678 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 03:03:20,678 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,679 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 03:03:20,679 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 03:03:20,679 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 03:03:20,681 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-653986615] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:20,724 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-816216791] 03:03:20,724 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 03:03:20,724 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1376310564] 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 03:03:20,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 03:03:20,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 03:03:20,981 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 03:03:21,279 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 03:03:21,279 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,279 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 03:03:21,511 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:21,511 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:21,512 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:21,512 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,512 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:21,512 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,512 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,512 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,517 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:21,517 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,517 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,517 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:21,517 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,517 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,579 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 03:03:21,579 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,579 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 03:03:21,579 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,581 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:21,581 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 03:03:21,581 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:21,581 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:21,581 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=null 03:03:21,582 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:21,582 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,582 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 03:03:21,583 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 03:03:21,583 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 03:03:21,584 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 03:03:21,584 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,584 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 03:03:21,585 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 03:03:21,585 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:21,585 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 03:03:21,585 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:21,585 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:21,585 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 03:03:21,592 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:21,629 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 03:03:21,630 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1530589622] 03:03:21,630 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:21,630 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Queuing TransactionOperation 03:03:21,630 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 03:03:21,630 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 03:03:21,630 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars 03:03:21,630 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 03:03:21,630 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 03:03:21,632 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, disabling transaction tracking 03:03:21,632 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 03:03:21,632 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 03:03:21,632 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:21,632 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:21,632 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,632 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 03:03:21,632 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Creating transaction : member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 03:03:21,633 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#-540469034] 03:03:21,633 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#-540469034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:21,633 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 03:03:21,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 03:03:21,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Number of entries to be appended = 1 03:03:21,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:21,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 03:03:21,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 03:03:21,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Log size is now 1 03:03:21,635 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:21,636 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#-540469034, transactionId=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0, version=13] 03:03:21,636 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 03:03:21,636 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:21,636 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#-540469034)] 03:03:21,636 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done true 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: 0, nextIndex: 1 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Setting last applied to 0 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=4920663} is already disabled 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:21,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 03:03:21,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Commit index set to 0 03:03:21,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:21,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Setting last applied to 0 03:03:21,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 03:03:21,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 03:03:21,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:03:21,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 03:03:21,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 03:03:21,638 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 ReadData operation succeeded 03:03:21,639 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 03:03:21,640 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,640 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-265: Closing 03:03:21,640 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 03:03:21,641 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 03:03:21,641 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:21,641 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:21,645 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:21,646 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:21,646 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:21,646 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:21,648 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:21,662 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 03:03:21,663 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithShardLeaderNotResponding commit rateLimit adjusted to 71.23365547131583 03:03:21,663 PM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 03:03:21,663 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:21,663 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:21,663 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 03:03:21,663 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 03:03:21,663 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 03:03:21,664 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying 1 components for commit 03:03:21,664 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying transaction for shard cars 03:03:21,664 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 03:03:21,664 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 canCommit 03:03:21,664 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:21,664 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:21,728 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:03:21,757 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:03:21,757 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 03:03:21,757 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 03:03:21,827 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:03:21,857 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:03:21,857 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 03:03:21,857 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 03:03:21,927 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:03:21,957 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:03:21,957 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 03:03:21,957 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 03:03:22,27 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 03:03:22,57 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 03:03:22,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 03:03:22,58 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 03:03:22,127 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Creating NoOpTransaction because of error akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:22,128 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:22,128 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.ShardLeaderNotRespondingException: Could not create a READ_WRITE transaction on shard cars. The shard leader isn't responding. at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.createTransactionContext(RemoteTransactionContextSupport.java:213) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.onCreateTransactionComplete(RemoteTransactionContextSupport.java:195) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport$1.onComplete(RemoteTransactionContextSupport.java:136) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:22,129 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:03:22,129 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:03:22,129 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 03:03:22,129 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:03:22,129 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:03:22,130 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 03:03:22,130 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 03:03:22,130 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:03:22,130 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 03:03:22,131 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 03:03:22,131 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 03:03:22,132 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:22,132 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:22,662 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6859000036478932141] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:23,152 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4611601372356322463] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:23,153 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:23,153 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:23,154 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:23.156] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:23,170 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:23,170 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:23,172 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:23.174] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:23,210 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:23,219 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-193558500452958537] 03:03:23,219 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:23,225 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:23,225 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:23,226 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:23,226 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:23,243 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 closeTransaction called 03:03:23,243 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_33#-540469034)] 03:03:23,246 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:23,246 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:23,246 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:23,246 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:23,252 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3727354107375897672] 03:03:23,253 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:23,254 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:23,254 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:23,260 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:23,260 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:23,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:23,282 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-3707644485405436885] 03:03:23,282 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:23,288 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:23,288 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:23,294 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:23,294 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:23,302 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:23,303 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:23,305 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 03:03:23,320 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1958586831] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:23,321 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:23,323 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:23,325 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:23,331 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 03:03:23,331 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:03:23,331 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@577e2b24 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@127cafaf 03:03:23,331 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:23,331 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,331 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:23,331 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,333 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7d17c954 03:03:23,333 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709823803331, None)) 03:03:23,333 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823803330,0,0)) 03:03:23,333 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@127cafaf to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5a5862c 03:03:23,333 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7e36c5dc 03:03:23,334 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 03:03:23,335 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:23,337 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:23,340 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 03:03:23,341 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:23,341 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,341 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,341 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,341 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1980671916] 03:03:23,341 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 03:03:23,342 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1980671916] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:23,342 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 03:03:23,344 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1640036623] 03:03:23,345 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#279974755 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,345 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1640036623] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:23,345 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:23,345 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:23,345 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:03:23,345 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:03:23,345 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#279974755] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:23,345 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 03:03:23,345 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 03:03:23,345 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 03:03:23,345 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 03:03:23,346 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 03:03:23,361 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:23,362 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:23,364 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1194312584] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:23,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 03:03:23,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:03:23,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5f98773b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@493c5b23 03:03:23,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2cb4df44 03:03:23,364 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709823803364, None)) 03:03:23,364 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:23,365 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823803363,0,0)) 03:03:23,365 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@493c5b23 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@703748ef 03:03:23,365 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 03:03:23,365 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:23,365 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,366 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:23,366 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@609f7c69 03:03:23,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:23,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,366 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,366 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611] 03:03:23,367 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 03:03:23,367 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:23,367 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1618362336 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,367 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-1926965835] 03:03:23,367 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1618362336] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:23,367 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-1926965835] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:23,367 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 03:03:23,368 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:23,368 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:23,368 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 03:03:23,368 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 03:03:23,368 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:03:23,368 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:03:23,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 03:03:23,368 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 03:03:23,368 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 03:03:23,416 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1958586831] 03:03:23,416 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 03:03:23,416 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1980671916] 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:03:23,576 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:03:23,576 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:03:23,866 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:03:23,866 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:23,866 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 03:03:23,984 PM [cluster-test-shard-dispatcher-19] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:24,116 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:03:24,116 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,116 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 03:03:24,237 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:24,237 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:24,238 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:24,238 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,238 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,238 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:24,238 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,238 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,238 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,244 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:24,244 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,244 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,244 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:24,244 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,244 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,308 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:24,346 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:03:24,346 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,346 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 03:03:24,346 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,348 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:24,348 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:03:24,348 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:24,348 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:24,348 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=null 03:03:24,348 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:24,348 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,348 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:24,349 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:24,349 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:24,349 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:03:24,349 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:24,349 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:24,349 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 03:03:24,350 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 03:03:24,350 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:03:24,350 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:24,350 PM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:24,350 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 03:03:24,350 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 03:03:24,350 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:03:24,350 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:24,350 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,350 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 03:03:24,351 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:03:24,351 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:24,351 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:24,351 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:03:24,352 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 03:03:24,352 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:24,373 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleShardTransactionsWithLeaderChanges commit rateLimit adjusted to 71.23365547131583 03:03:24,373 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,373 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1194312584] 03:03:24,373 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 03:03:24,373 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:24,373 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 03:03:24,373 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying 1 components for commit 03:03:24,373 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying transaction for shard cars 03:03:24,373 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 03:03:24,373 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 canCommit 03:03:24,373 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 03:03:24,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 03:03:24,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] for shard cars 03:03:24,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] found - creating WRITE_ONLY transaction context 03:03:24,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:24,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 03:03:24,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 directCommit called 03:03:24,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 sending 2 batched modifications, ready: true 03:03:24,374 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, disabling transaction tracking 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying 2 batched modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Readying Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 of 2 operations, client version 13 03:03:24,376 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 ready 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 validated 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: success 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 candidate ready 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - messages: [] 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 prepared 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,377 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 submitted to persistence 03:03:24,378 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 03:03:24,378 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:24,378 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 03:03:24,378 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 03:03:24,378 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:24,378 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 03:03:24,378 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 03:03:24,378 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 03:03:24,378 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:24,379 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 03:03:24,379 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=5143418} is already disabled 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 03:03:24,381 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=181} 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:24,382 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 1 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges$a] 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1336856269] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 4 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:24,383 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 03:03:24,384 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1336856269] 03:03:24,384 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 03:03:24,384 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 81.49 μs 03:03:24,384 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1336856269] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:24,384 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 successfully completed direct commit 03:03:24,384 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=13] 03:03:24,384 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 1 03:03:24,384 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:24,384 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 03:03:24,384 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 03:03:24,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 03:03:24,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,385 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 03:03:24,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-820148944] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 03:03:24,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 4 03:03:24,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 03:03:24,386 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,386 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:03:24,386 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,386 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:03:24,386 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 03:03:24,386 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:03:24,386 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-820148944] 03:03:24,386 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 03:03:24,386 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 79.41 μs 03:03:24,386 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-820148944] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:24,386 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:24,386 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,386 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-270: Closing 03:03:24,386 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 03:03:24,387 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:24,390 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:24,390 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:24,394 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:24,394 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:24,394 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:24,394 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:24,396 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:24,412 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 03:03:24,457 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1194312584] 03:03:24,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 03:03:24,457 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611] 03:03:24,457 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,543 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 03:03:24,543 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:03:24,544 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:24,544 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:03:24,545 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 03:03:24,545 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:24,545 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:24,545 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:03:24,565 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:24,573 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [382779459851742285] 03:03:24,574 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:24,575 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:24,575 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:24,576 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:24,576 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:24,586 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:24,588 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:24,589 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 03:03:24,594 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:24,595 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1021454833] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:24,597 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:03:24,597 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 03:03:24,597 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@63ab0e61 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@54a01e59 03:03:24,597 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:24,598 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@17c56c58 03:03:24,598 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,598 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709823804598, None)) 03:03:24,598 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823804597,0,0)) 03:03:24,598 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@20b876e3 03:03:24,598 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@54a01e59 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4686733f 03:03:24,598 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 03:03:24,598 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:24,598 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,599 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 03:03:24,599 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-441209182] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:24,600 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-984002803 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,600 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-984002803] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:24,600 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1806019034] 03:03:24,600 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-1806019034] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:24,600 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 71 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 181 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 1 to 1 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@42aa2478}, purged=MutableUnsignedLongSet{size=0}} 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 510.0 μs - Switching actor to Follower - last log index = 1, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 2 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:03:24,602 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:03:24,603 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 03:03:24,603 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 03:03:24,603 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 03:03:24,603 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 03:03:24,603 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 03:03:24,649 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1194312584] 03:03:24,649 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 03:03:24,649 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611] 03:03:24,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:03:24,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,694 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 03:03:24,694 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:03:24,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:24,874 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 03:03:24,874 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,44 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:03:25,44 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,44 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 03:03:25,44 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:03:25,172 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:03:25,172 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:03:25,214 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:03:25,214 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,214 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 03:03:25,214 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,405 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-193558500452958537] but current is [382779459851742285]. Reason: ActorSystem terminated 03:03:25,407 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-193558500452958537] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:25,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:03:25,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,424 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 03:03:25,424 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,462 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 5 03:03:25,462 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than Candidate's term 5 - switching to Follower 03:03:25,462 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 6 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:03:25,463 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 03:03:25,463 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] LocalActorRefProvider$$anon$2 - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [-3727354107375897672]. This system with UID [382779459851742285] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 6 - updating term 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 4 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,463 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=true] 03:03:25,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:25,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:03:25,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:03:25,464 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 7 - updating term 03:03:25,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 03:03:25,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 4 03:03:25,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,464 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=true] 03:03:25,465 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:25,465 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:03:25,465 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:03:25,465 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 8 - updating term 03:03:25,465 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 03:03:25,465 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 4 03:03:25,465 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,465 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=true] 03:03:25,466 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 10 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 10 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 03:03:25,466 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 03:03:25,466 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:03:25,466 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:25,466 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:03:25,466 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:03:25,466 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 9 - updating term 03:03:25,466 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 03:03:25,467 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 4 03:03:25,467 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,467 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=true] 03:03:25,467 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 03:03:25,469 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:25,472 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:25,472 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = null 03:03:25,472 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 03:03:25,472 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:03:25,472 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:25,472 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:25,472 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 03:03:25,473 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 03:03:25,473 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:25,473 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - New leader address: akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,473 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,474 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 10 03:03:25,474 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 03:03:25,503 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,503 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1194312584] 03:03:25,503 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 03:03:25,503 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying 1 components for commit 03:03:25,503 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying transaction for shard cars 03:03:25,503 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 03:03:25,504 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 canCommit 03:03:25,504 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:25,504 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7c52c0d3] 03:03:25,504 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7c52c0d3] 03:03:25,504 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611)] for shard cars 03:03:25,504 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:03:25,504 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@379ee02c 03:03:25,505 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 = [ 1 ] 03:03:25,505 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611)] 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: handleReadyLocalTransaction for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying local modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 validated 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: success 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 candidate ready 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - candidate rootPath: / 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - messages: [] 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 prepared 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,505 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}]]] 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 submitted to persistence 03:03:25,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 03:03:25,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}]]] 03:03:25,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 03:03:25,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:25,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=240} 03:03:25,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 03:03:25,508 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 03:03:25,508 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=10, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=10, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:25,508 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 2 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data Simple{size=240} 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: commit - successfulFromPrevious: [] 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges%23-569226611$b] 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 10 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:25,510 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 03:03:25,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 successfully completed direct commit 03:03:25,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed successfully 03:03:25,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 03:03:25,511 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 86.02 μs 03:03:25,511 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 03:03:25,511 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:25,511 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@379ee02c 03:03:25,511 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 03:03:25,511 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 03:03:25,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,512 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 03:03:25,512 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 03:03:25,513 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:03:25,513 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 03:03:25,513 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 03:03:25,513 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#-569226611] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-441209182] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-441209182] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:25,513 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:25,513 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:25,514 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:03:25,514 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 03:03:25,514 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:03:25,514 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:25,514 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:25,514 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:03:25,514 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:03:25,514 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:25,515 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1021454833] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1021454833] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:25,515 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-2002453102] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-2002453102] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:25,515 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1021454833] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#1021454833] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:25,516 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#-1383401582]] intended for system uid [-193558500452958537], self uid is [382779459851742285] 03:03:25,516 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:25,517 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:25,522 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:25,522 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:25,523 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:25,523 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:25,525 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:25,534 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:03:25,534 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:03:25,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-273: Closing 03:03:25,534 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 03:03:25,535 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:03:25,535 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:25,535 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:26,349 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 03:03:26,350 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-193558500452958537] but current is [382779459851742285]. Reason: Cluster member removed, previous status [Down] 03:03:26,537 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [382779459851742285] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:26,556 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:26,556 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:26,557 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:26,558 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3727354107375897672] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/07/2024 15:03:26.559] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:26,565 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:26,565 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:26,568 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:26.570] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:26,600 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:26,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4450886763862515931] 03:03:26,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:26,608 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:26,608 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:26,616 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:26,616 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:26,621 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:26,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:26,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:26,627 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:26,628 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-1247950473916758444] 03:03:26,628 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:26,630 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:26,630 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:26,640 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:26,640 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:26,657 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:03:26,671 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8828955238710049438] 03:03:26,672 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:26,673 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:26,673 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:26,683 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:26,683 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:26,685 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:26,687 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:26,688 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 03:03:26,694 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 03:03:26,697 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:26,705 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 03:03:26,705 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 03:03:26,705 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:03:26,705 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,705 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5ce8fcae to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@795f468e 03:03:26,705 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:26,705 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,711 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6cd862d0 03:03:26,711 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709823806709, None)) 03:03:26,715 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823806708,0,0)) 03:03:26,716 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@795f468e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@19d5f4d0 03:03:26,716 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@418921b 03:03:26,716 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 03:03:26,716 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:26,718 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:26,720 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 03:03:26,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:26,721 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,739 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,739 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,739 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] 03:03:26,740 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 03:03:26,741 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 03:03:26,741 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:26,741 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#205260747 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,741 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#205260747] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:26,741 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#-253460738] 03:03:26,742 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#-253460738] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:26,746 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:26,746 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:26,747 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 03:03:26,747 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 03:03:26,747 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:26,747 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 03:03:26,748 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 03:03:26,748 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 03:03:26,748 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 03:03:26,765 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:26,771 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1865696109] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:26,772 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 03:03:26,775 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:26,781 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:26,782 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:26,783 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 03:03:26,783 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:03:26,783 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@34625676 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@a2e6f69 03:03:26,784 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@59ce755 03:03:26,784 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 03:03:26,784 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:26,784 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709823806784, None)) 03:03:26,784 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823806783,0,0)) 03:03:26,784 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:26,784 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,784 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3dd3c10c 03:03:26,784 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@a2e6f69 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@b29957d 03:03:26,784 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 03:03:26,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:26,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:26,785 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:26,785 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1093670469] 03:03:26,786 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 03:03:26,786 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 03:03:26,786 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1093670469] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:26,786 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#421832976] 03:03:26,786 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#2058497436 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:26,787 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#2058497436] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:26,787 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#421832976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:26,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:26,787 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:26,787 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 03:03:26,787 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 03:03:26,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:26,787 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 03:03:26,787 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 03:03:26,787 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:26,787 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 03:03:26,787 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 03:03:26,835 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] 03:03:26,835 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 03:03:26,835 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 03:03:27,4 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 03:03:27,4 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 03:03:27,244 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 03:03:27,244 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,244 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 03:03:27,330 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:27,534 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 03:03:27,534 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,534 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 03:03:27,635 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:27,635 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:27,636 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:27,636 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 03:03:27,636 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,637 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:27,637 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,637 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,637 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,643 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:27,643 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 03:03:27,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,643 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:27,643 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,643 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,694 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:27,774 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 03:03:27,774 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,774 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 03:03:27,774 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,776 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:27,776 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 03:03:27,776 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:27,776 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:27,776 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 03:03:27,776 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:27,776 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,776 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:27,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:27,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:27,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 03:03:27,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:27,778 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:27,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:03:27,778 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 03:03:27,779 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 03:03:27,779 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 03:03:27,779 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:27,779 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:27,779 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 03:03:27,779 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 03:03:27,780 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 03:03:27,780 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,780 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 03:03:27,780 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 03:03:27,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:27,780 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:27,780 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 03:03:27,781 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 03:03:27,781 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:27,796 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithIsolatedLeader commit rateLimit adjusted to 71.23365547131583 03:03:27,796 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:27,796 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] 03:03:27,796 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Queuing TransactionOperation 03:03:27,796 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 03:03:27,796 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] 03:03:27,796 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Queuing TransactionOperation 03:03:27,796 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:27,796 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] 03:03:27,796 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Queuing TransactionOperation 03:03:27,797 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:27,797 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6af18ffe] 03:03:27,797 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:27,797 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6af18ffe] 03:03:27,797 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:27,797 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6af18ffe] 03:03:27,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6af18ffe] 03:03:27,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188)] for shard cars 03:03:27,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 03:03:27,797 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@f428994 03:03:27,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6af18ffe] 03:03:27,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188)] for shard cars 03:03:27,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@f428994 03:03:27,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6af18ffe] 03:03:27,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188)] for shard cars 03:03:27,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@f428994 03:03:27,807 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:03:27,807 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,807 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 03:03:27,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 03:03:27,808 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 03:03:27,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 03:03:27,808 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 03:03:27,808 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 03:03:27,808 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:03:27,809 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying 1 components for commit 03:03:27,809 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 03:03:27,809 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying transaction for shard cars 03:03:27,809 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 = [ 1 ] 03:03:27,809 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188)] 03:03:27,809 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1865696109] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1865696109] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:27,809 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:03:27,809 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1865696109] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1865696109] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:27,809 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-712114844] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-712114844] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:27,809 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:27,810 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, disabling transaction tracking 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}]]] 03:03:27,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:27,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 validated 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:27,810 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: success 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:27,810 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 candidate ready 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - messages: [] 03:03:27,811 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 prepared 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 submitted to persistence 03:03:27,811 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 03:03:27,812 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:27,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 03:03:27,812 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 03:03:27,862 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] 03:03:27,894 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:27,913 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] 03:03:27,964 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] 03:03:28,14 PM [cluster-test-shard-dispatcher-11] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 03:03:28,15 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] 03:03:28,16 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:28,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 03:03:28,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:28,16 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 03:03:28,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-278: Closing 03:03:28,16 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:03:28,16 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 03:03:28,16 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 03:03:28,17 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying 1 components for commit 03:03:28,17 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying transaction for shard cars 03:03:28,17 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 = [ 1 ] 03:03:28,17 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188)] 03:03:28,17 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:28,17 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 03:03:28,17 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@7f24be72 to retry 03:03:28,18 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 canCommit 03:03:28,435 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Timer expired for message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@7f24be72 03:03:28,435 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] LocalThreePhaseCommitCohort - Failed to prepare transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 on backend org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Could not process ready local transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0. Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport$MessageInfo.timedOut(ShardTransactionMessageRetrySupport.java:102) at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport.onTimerMessage(ShardTransactionMessageRetrySupport.java:72) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:410) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:03:28,436 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 03:03:28,436 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying 1 components for commit 03:03:28,436 PM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying transaction for shard cars 03:03:28,436 PM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 = [ 1 ] 03:03:28,436 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1881302188)] 03:03:28,437 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:28,437 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,437 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@12e26384 to retry 03:03:28,438 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:28,438 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 03:03:28,439 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:28,442 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 03:03:28,443 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:03:28,443 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 03:03:28,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 03:03:28,444 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 03:03:28,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@723a80ee to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d34a0d8 03:03:28,444 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1153373 03:03:28,444 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709823808444, None)) 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:28,445 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823808443,0,0)) 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,445 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d34a0d8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@8811d19 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2f652191 03:03:28,445 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 03:03:28,445 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 canCommit 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:28,445 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,446 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1841539322] 03:03:28,446 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,446 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,446 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1841539322] 03:03:28,446 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 03:03:28,446 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,446 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#1841539322] 03:03:28,446 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 03:03:28,446 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#991406434 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,446 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 03:03:28,446 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#253892816] 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:28,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 03:03:28,447 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 03:03:28,447 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:28,447 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 03:03:28,447 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 03:03:28,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,495 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=170} 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 03:03:28,497 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:28,497 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 03:03:28,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:28,498 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:28,498 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 03:03:28,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 03:03:28,499 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 03:03:28,499 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 1, nextIndex: 2 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 0 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=692113169} is already disabled 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 1 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:28,502 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-1881302188$a] 03:03:28,502 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 successfully completed direct commit 03:03:28,503 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed successfully 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1218895729] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 03:03:28,503 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1218895729] 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 03:03:28,503 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 03:03:28,503 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1218895729] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:28,503 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 141.6 μs 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-279: Closing 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Retrying message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@12e26384 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 03:03:28,503 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 validated 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: success 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,504 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 candidate ready 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - candidate rootPath: / 03:03:28,504 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 03:03:28,504 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - messages: [] 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 prepared 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 68 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 03:03:28,504 PM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 canCommit 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 03:03:28,504 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 submitted to persistence 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 1 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 03:03:28,505 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 03:03:28,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 03:03:28,506 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:03:28,506 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1970642407] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 03:03:28,506 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1970642407] 03:03:28,506 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 03:03:28,506 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 03:03:28,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 03:03:28,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 03:03:28,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:28,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=68} 03:03:28,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 03:03:28,507 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 108.5 μs 03:03:28,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 03:03:28,507 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:28,507 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 03:03:28,507 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 03:03:28,507 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 2, nextIndex: 3 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: commit - successfulFromPrevious: [] 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-1881302188$b] 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:28,515 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 03:03:28,515 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 successfully completed direct commit 03:03:28,516 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed successfully 03:03:28,516 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 03:03:28,516 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:03:28,516 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 13.41 μs 03:03:28,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:28,516 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 03:03:28,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 03:03:28,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 03:03:28,516 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 03:03:28,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:28,516 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:28,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@182141d9 03:03:28,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@182141d9 03:03:28,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 03:03:28,516 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:28,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 2 03:03:28,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:03:28,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 03:03:28,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 03:03:28,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 03:03:28,517 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:03:28,518 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 03:03:28,518 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 03:03:28,518 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 03:03:28,518 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 39.93 μs 03:03:28,518 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader] 03:03:28,518 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 03:03:28,518 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:28,518 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:28,520 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 2, lastIndex: 2, isVoting: true 03:03:28,520 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Follower's log matches - sending ElectionTimeout 03:03:28,520 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:28,520 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leader transfer complete 03:03:28,521 PM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:28,655 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:03:30,535 PM [cluster-test-shard-dispatcher-20] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Successfully transferred leadership to member-1-shard-cars-testTransactionWithIsolatedLeader in 2.018 s 03:03:30,535 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer succeeded - sending PoisonPill 03:03:30,536 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 03:03:30,536 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-282: Closing 03:03:30,536 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 03:03:30,536 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 03:03:30,536 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 03:03:30,537 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:03:30,537 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:03:30,537 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 03:03:30,537 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:03:30,537 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:30,537 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#1689918138] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1689918138] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:30,537 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:03:30,537 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:30,537 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1391251099] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:30,538 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#1689918138] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1689918138] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:30,538 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:30,538 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:30,544 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:30,544 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:30,545 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:30,545 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:30,566 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:03:30,567 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:03:30,567 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 03:03:30,567 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 03:03:30,567 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 03:03:30,568 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:30,568 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:31,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4450886763862515931] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:31,583 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:31,585 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:31,585 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-1247950473916758444] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/07/2024 15:03:31.588] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:31,606 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:31,606 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:03:31.612] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:31,640 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:31,647 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4654109475158656594] 03:03:31,647 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:31,649 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:31,649 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:31,650 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:31,650 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:31,671 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:31,671 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:31,671 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:31,675 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:31,686 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6387856497715933192] 03:03:31,686 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:31,688 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:31,688 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:31,689 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:31,689 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:31,705 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:03:31,717 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4036425519235231250] 03:03:31,717 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:31,726 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:31,726 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:31,726 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:31,726 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:31,730 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:31,732 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:31,734 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 03:03:31,741 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 03:03:31,748 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#15735124] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,750 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 03:03:31,750 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:03:31,750 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2155ad64 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7a6677a8 03:03:31,750 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@593eb80c 03:03:31,751 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1709823811750, None)) 03:03:31,751 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 03:03:31,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,751 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823811749,0,0)) 03:03:31,751 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:31,751 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7a6677a8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4410b7bf 03:03:31,752 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 03:03:31,753 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:31,756 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:31,759 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 03:03:31,761 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:31,761 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,761 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:31,761 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1de91c6 03:03:31,761 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:31,761 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,762 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,762 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,762 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-1-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133] 03:03:31,762 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:31,762 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 03:03:31,762 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,763 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#485148405 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,764 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#485148405] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,764 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1498837123] 03:03:31,764 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1498837123] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,764 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 03:03:31,764 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 03:03:31,764 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#197416271] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,765 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1496305142] 03:03:31,765 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-553304571 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:31,765 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-553304571] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,765 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1496305142] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,765 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 03:03:31,769 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 03:03:31,769 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#438327366] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,770 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:31,770 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:31,770 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-1-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#197416271] 03:03:31,771 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:31,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:31,771 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:31,771 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:31,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:31,772 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:31,772 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 03:03:31,772 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 03:03:31,772 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 03:03:31,772 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 03:03:31,772 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 03:03:31,772 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 03:03:31,772 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 03:03:31,772 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 03:03:31,772 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 03:03:31,772 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 03:03:31,772 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 03:03:31,772 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 03:03:31,773 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 03:03:31,773 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,773 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:31,774 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 03:03:31,774 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:03:31,775 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3bf4db16 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@46738fb4 03:03:31,775 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2b4e5bf5 03:03:31,775 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1709823811775, None)) 03:03:31,776 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823811774,0,0)) 03:03:31,776 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@46738fb4 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@67b97f0 03:03:31,776 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3db658f7 03:03:31,776 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 03:03:31,776 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:31,776 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,777 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:31,778 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 03:03:31,778 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-860893785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,778 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 03:03:31,778 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-258331494 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,778 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-258331494] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,778 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 03:03:31,779 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:31,779 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 03:03:31,782 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:31,786 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#-1852874624] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,786 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1886555066] 03:03:31,786 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1623405960] 03:03:31,786 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1886555066] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,786 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1623405960] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,787 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#2040759836 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:31,787 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#2040759836] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:31,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:31,788 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:31,788 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:31,788 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:31,788 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:31,788 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 03:03:31,788 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 03:03:31,788 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 03:03:31,788 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:31,788 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 03:03:31,788 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 03:03:31,788 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 03:03:31,788 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 03:03:31,788 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 03:03:31,788 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 03:03:31,788 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 03:03:31,788 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 03:03:31,788 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 03:03:31,792 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:31,796 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:31,797 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:31,797 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,797 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,797 PM [cluster-test-shard-dispatcher-33] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:31,797 PM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:31,797 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,797 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:31,827 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#15735124] 03:03:31,827 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 03:03:31,827 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133] 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 03:03:31,988 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 03:03:31,988 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 03:03:32,68 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 03:03:32,68 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 03:03:32,218 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:32,218 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,218 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 03:03:32,322 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:32,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:32,337 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,337 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 03:03:32,382 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:32,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:32,468 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,468 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 03:03:32,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:32,577 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,577 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 03:03:32,668 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:32,668 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:32,668 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:32,668 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,668 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,668 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 03:03:32,668 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,668 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:32,668 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,668 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,669 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:32,669 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,669 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,669 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:32,727 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:32,732 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:32,778 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:32,778 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,778 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 03:03:32,778 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,780 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:32,780 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 03:03:32,780 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:32,780 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:32,780 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 03:03:32,780 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:32,780 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,780 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:32,782 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:32,782 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:32,782 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 03:03:32,782 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:32,782 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:32,782 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:03:32,783 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 03:03:32,783 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 03:03:32,783 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:03:32,783 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:32,783 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:32,783 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 03:03:32,784 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 03:03:32,784 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,784 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 03:03:32,784 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:03:32,784 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:32,784 PM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:32,784 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:03:32,784 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:32,833 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#15735124] 03:03:32,834 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 03:03:32,834 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#197416271] 03:03:32,887 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:03:32,887 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,887 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 03:03:32,887 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:32,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:32,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 03:03:32,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:32,890 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:32,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 03:03:32,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:32,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,890 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 03:03:32,892 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 03:03:32,892 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 03:03:32,893 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 03:03:32,893 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,894 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 03:03:32,894 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:03:32,894 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:32,894 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:32,894 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 03:03:32,895 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:03:32,895 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:32,898 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 03:03:32,935 PM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithMultipleShards commit rateLimit adjusted to 71.23365547131583 03:03:32,935 PM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,935 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#438327366] 03:03:32,936 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:03:32,936 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#438327366] 03:03:32,936 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:03:32,936 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 03:03:32,936 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 03:03:32,936 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:03:32,936 PM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 03:03:32,936 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 03:03:32,936 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 03:03:32,936 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:32,936 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 03:03:32,936 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:03:32,936 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 03:03:32,936 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 03:03:32,936 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars 03:03:32,936 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 03:03:32,936 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 03:03:32,937 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 03:03:32,939 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 13 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,940 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 03:03:32,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,940 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:32,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:32,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:32,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 03:03:32,941 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 13 03:03:32,941 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 03:03:32,943 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133)] resolved 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=13] 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#197416271)] resolved 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133)] 03:03:32,944 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133)] 03:03:32,946 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,946 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 03:03:32,946 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,946 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:32,946 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 03:03:32,946 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 03:03:32,946 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:32,946 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:32,946 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 03:03:32,946 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:32,946 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:32,946 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 03:03:32,946 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 03:03:32,946 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,946 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,946 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 03:03:32,946 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 03:03:32,946 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:32,946 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:32,948 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:03:32,948 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#197416271)] 03:03:32,948 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#197416271)] 03:03:32,948 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,948 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 03:03:32,949 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=9947858} is already disabled 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=9248080} is already disabled 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:32,949 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:32,949 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:32,950 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:32,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 03:03:32,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:32,951 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 03:03:32,952 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 03:03:32,952 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:03:32,952 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:32,952 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 03:03:32,952 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 03:03:32,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 03:03:32,952 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:03:32,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:32,952 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 03:03:32,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 03:03:32,952 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:03:32,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:03:32,952 PM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:03:32,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 03:03:32,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:03:32,952 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:32,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 03:03:32,952 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:32,952 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 03:03:32,952 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 03:03:32,952 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133)] 03:03:32,952 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133)] 03:03:32,952 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#197416271)] 03:03:32,952 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#197416271)] 03:03:32,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 03:03:32,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:03:32,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 03:03:32,954 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:32,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 03:03:32,954 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,955 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 03:03:32,955 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:03:32,955 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:03:32,955 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:32,955 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:32,955 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 175 03:03:32,955 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 03:03:32,955 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 03:03:32,955 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 03:03:32,957 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:03:32,957 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 03:03:32,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 03:03:32,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:32,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:32,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 03:03:32,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:03:32,959 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=175} 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:32,960 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 03:03:32,960 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-cars-testWriteTransactionWithMultipleShards%23-1937186133$e] 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-people-testWriteTransactionWithMultipleShards%23197416271$f] 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#420124771] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#1870707804] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:32,961 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:32,961 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:32,962 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:32,962 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#420124771] 03:03:32,962 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:03:32,962 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 87.14 μs 03:03:32,962 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#420124771] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:32,962 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#1870707804] 03:03:32,962 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:03:32,962 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#1870707804] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:32,963 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 126.7 μs 03:03:32,965 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 03:03:32,965 PM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:03:32,965 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 03:03:32,965 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:32,965 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - trying create transaction 03:03:32,965 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 03:03:32,965 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 03:03:32,966 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:03:32,967 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#-291302607, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 03:03:32,967 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 03:03:32,967 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#-291302607)] 03:03:32,968 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#-291302607] 03:03:32,968 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#-291302607] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:32,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 03:03:32,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:32,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 03:03:32,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:03:32,968 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 03:03:32,968 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 03:03:32,968 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:32,968 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 03:03:32,968 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:03:32,968 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 03:03:32,969 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,969 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:03:32,969 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1496049604] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 03:03:32,969 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:03:32,969 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:32,969 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#701930506] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 03:03:32,969 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:03:32,970 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:32,970 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1496049604] 03:03:32,970 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#701930506] 03:03:32,970 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:03:32,970 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:03:32,970 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 97.48 μs 03:03:32,971 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1496049604] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:32,971 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#701930506] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:32,971 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 192.6 μs 03:03:32,972 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 03:03:32,972 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 03:03:32,972 PM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 03:03:32,972 PM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - trying create transaction 03:03:32,972 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 03:03:32,972 PM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 03:03:32,974 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:03:32,974 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-1437327921] 03:03:32,975 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-1437327921, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=13] 03:03:32,975 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 03:03:32,975 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-1437327921)] 03:03:32,976 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 03:03:32,977 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5388f05f 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5388f05f 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@704dbf4f 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@704dbf4f 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 03:03:32,977 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:32,978 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:32,978 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:32,978 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:32,978 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:32,978 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:32,978 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:32,979 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards] 03:03:32,979 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards] 03:03:32,979 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:32,979 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:32,979 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:32,979 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:32,979 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:32,979 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:32,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:03:32,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:03:32,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:32,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 03:03:32,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:03:32,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:03:32,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:32,980 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 03:03:33,388 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 03:03:33,388 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1937186133]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:33,388 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 03:03:33,388 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 03:03:33,388 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-290: Closing 03:03:33,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-289: Closing 03:03:33,388 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:03:33,388 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:03:33,389 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 03:03:33,389 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 03:03:33,389 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:03:33,389 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 03:03:33,390 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 03:03:33,390 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:33,390 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:33,397 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:33,398 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:33,398 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:33,398 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:33,400 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:33,409 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 03:03:33,410 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 03:03:33,410 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 03:03:33,410 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:03:33,412 PM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 03:03:33,412 PM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 03:03:33,412 PM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:03:33,412 PM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 03:03:33,412 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:33,413 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:34,411 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4654109475158656594] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:34,425 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:34,425 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:34,427 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:34.429] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:34,431 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-6387856497715933192] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:34,447 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:34,447 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:34,449 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:34.451] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:34,477 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:34,484 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7144674574868320386] 03:03:34,484 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:34,485 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:34,485 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:34,492 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:34,492 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:34,499 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:34,499 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:34,499 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:34,503 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:34,510 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3367855382343029589] 03:03:34,510 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:34,512 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:34,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:34,520 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:34,520 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:34,550 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:34,562 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2317101352899176061] 03:03:34,562 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:34,567 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:34,567 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:34,575 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:34,575 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:34,579 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:34,580 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:34,582 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 03:03:34,595 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:34,602 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:34,602 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-330966114] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,604 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 03:03:34,610 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 03:03:34,610 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 03:03:34,610 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:03:34,610 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,610 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6d6077dd to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b32e10b 03:03:34,610 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5c471dc6 03:03:34,610 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1709823814610, None)) 03:03:34,610 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,610 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823814609,0,0)) 03:03:34,611 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:34,611 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,611 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,611 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b32e10b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@d4f8892 03:03:34,611 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3f12edb 03:03:34,611 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 03:03:34,612 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:34,612 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,612 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:34,613 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:34,616 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 03:03:34,619 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124] 03:03:34,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,620 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 03:03:34,620 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 03:03:34,621 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 03:03:34,621 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 03:03:34,622 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,622 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,622 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#2108484156 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,622 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-903298859] 03:03:34,622 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#2108484156] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,622 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-903298859] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,622 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#961875211] 03:03:34,622 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#-1976809877] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,624 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,624 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,624 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488] 03:03:34,625 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:34,625 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:34,625 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 03:03:34,625 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 03:03:34,625 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:34,625 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 03:03:34,627 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1789856868] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,627 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 03:03:34,630 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#-1976809877 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,630 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 03:03:34,630 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 03:03:34,630 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 03:03:34,631 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:34,631 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:34,631 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 03:03:34,631 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 03:03:34,631 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:34,631 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 03:03:34,631 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 03:03:34,632 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 03:03:34,632 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 03:03:34,634 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#961875211] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,642 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:34,645 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:34,649 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 03:03:34,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:34,649 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 03:03:34,650 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:34,650 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,650 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,650 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 03:03:34,650 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:03:34,650 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@514cf9f3 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@75733d3c 03:03:34,650 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@29fc780d 03:03:34,650 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1709823814650, None)) 03:03:34,651 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823814649,0,0)) 03:03:34,651 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@68606b9b 03:03:34,651 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@75733d3c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@226ed32c 03:03:34,651 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-2-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1624811286] 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 03:03:34,652 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:03:34,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-2-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#-1241897043] 03:03:34,652 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1624811286] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,653 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 03:03:34,653 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#-1994376657 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:34,653 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#1024610285] 03:03:34,653 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#-1241897043] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,653 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1648290553 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 03:03:34,653 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-564405761] 03:03:34,654 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#-1994376657] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,654 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#1024610285] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,654 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#1648290553] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,654 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 03:03:34,654 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-564405761] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:34,656 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:34,656 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:34,656 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:34,656 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 03:03:34,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 03:03:34,656 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 03:03:34,656 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 03:03:34,656 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,656 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 03:03:34,656 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 03:03:34,657 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 03:03:34,659 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:34,659 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:34,659 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 03:03:34,659 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 03:03:34,659 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:34,659 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 03:03:34,659 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 03:03:34,659 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,660 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 03:03:34,660 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 03:03:34,702 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-330966114] 03:03:34,702 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 03:03:34,702 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124] 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:34,902 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 03:03:34,902 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 03:03:34,903 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 03:03:34,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:34,941 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:03:34,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 03:03:34,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:03:34,942 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 03:03:34,942 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:34,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 03:03:34,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:03:34,942 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 03:03:34,942 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 03:03:34,942 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 03:03:35,190 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:35,221 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:03:35,221 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:35,221 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 03:03:35,229 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:35,231 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:03:35,231 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:03:35,231 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 03:03:35,491 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:03:35,491 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:35,491 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 03:03:35,501 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:03:35,501 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:03:35,501 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:03:35,502 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:35,502 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:35,503 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:35,503 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:35,503 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:03:35,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 03:03:35,587 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:35,741 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:03:35,741 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:35,741 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 03:03:35,741 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:35,742 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 03:03:35,742 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:03:35,742 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 03:03:35,742 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 03:03:35,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:35,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:35,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 03:03:35,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 03:03:35,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:35,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:35,744 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:35,744 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:35,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 03:03:35,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 03:03:35,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:35,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:35,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 03:03:35,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:35,744 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:35,744 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:03:35,746 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 03:03:35,746 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 03:03:35,746 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 03:03:35,746 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 03:03:35,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:35,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:35,747 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 03:03:35,747 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 03:03:35,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:35,747 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:35,747 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 03:03:35,747 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 03:03:35,749 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 03:03:35,749 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 03:03:35,749 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 03:03:35,749 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:35,749 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 03:03:35,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:35,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:35,749 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=13] 03:03:35,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:35,749 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:35,749 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 03:03:35,750 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 03:03:35,750 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 03:03:35,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:03:35,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:35,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:03:35,750 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:35,810 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-330966114] 03:03:35,810 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 03:03:35,810 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488] 03:03:35,811 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf 03:03:35,812 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf changed state from IDLE to TX_OPEN 03:03:35,824 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 03:03:35,824 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:35,824 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:35,824 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1789856868] 03:03:35,824 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:35,824 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 03:03:35,824 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=13] 03:03:35,828 PM [ForkJoinPool.commonPool-worker-1] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3bdb3bf1, attempting to connect 03:03:35,829 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 03:03:35,838 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:03:35,842 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 03:03:35,842 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=33180, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:03:35,842 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:03:35,843 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:03:35,843 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 03:03:35,843 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:03:35,843 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:03:35,843 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1789856868] 03:03:35,843 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:03:35,843 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 03:03:35,843 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 03:03:35,843 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 03:03:35,843 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:03:35,843 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 03:03:35,843 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=13] 03:03:35,843 PM [ForkJoinPool.commonPool-worker-1] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2c53e777, attempting to connect 03:03:35,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@8361baf 03:03:35,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@1edc7526 03:03:35,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 03:03:35,845 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=6030, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:03:35,845 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf changed state from TX_OPEN to IDLE 03:03:35,845 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:03:35,845 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 03:03:35,845 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@341ce4d9 03:03:35,845 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 03:03:35,845 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@2b2f3983 03:03:35,848 PM [ForkJoinPool.commonPool-worker-1] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:03:35,848 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:03:35,849 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:03:35,850 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:35,850 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@616a8d9b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e 03:03:35,852 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:03:35,854 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:03:35,854 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862243170242} 03:03:35,855 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e 03:03:35,855 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,855 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 03:03:35,855 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 03:03:35,855 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:03:35,856 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2043/0x00000008016a3198@4a8b347 03:03:35,856 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY}, enqueuedTicks=862249987013} 03:03:35,858 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY}} 03:03:35,862 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 409 bytes, total slices to send: 1 03:03:35,862 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,862 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 03:03:35,862 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 03:03:35,863 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@701f4bde 03:03:35,863 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=862251477290} 03:03:35,863 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}} 03:03:35,863 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 289 bytes, total slices to send: 1 03:03:35,863 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5b04e1f2 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@79facc92 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@59ea2caf, splicing queue 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@79facc92 queue spliced 0 messages 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 16.01 ms 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@12bc782a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862249234624} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2043/0x00000008016a3198@30ccd27 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY}, enqueuedTicks=862250055364} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY}} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 417 bytes, total slices to send: 1 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 03:03:35,865 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@534d2cc8 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=862251513091} 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}} 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 290 bytes, total slices to send: 1 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@388bf794 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5ea51b38 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@85738c0, splicing queue 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5ea51b38 queue spliced 0 messages 03:03:35,866 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 713.0 μs 03:03:35,868 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:35,868 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:35,868 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:35,868 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:35,868 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:03:35,868 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:03:35,869 PM [cluster-test-shard-dispatcher-30] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:35,869 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,869 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:35,869 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@10e55c28 03:03:35,869 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,869 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 ready 03:03:35,869 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:03:35,869 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 03:03:35,870 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 03:03:35,870 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 03:03:35,870 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,870 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 validated 03:03:35,870 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:35,870 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 03:03:35,873 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 03:03:35,873 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@4faa5b3c 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 ready 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 to ready 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 is already in state READY (READY) 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating canCommit 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 validated 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:35,873 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed canCommit 03:03:35,874 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:35,882 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 03:03:35,882 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:03:35,882 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY}, enqueuedTicks=862249987013, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 03:03:35,882 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 03:03:35,882 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 03:03:35,882 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 03:03:35,882 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=862251477290, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 03:03:35,882 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:35,882 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 03:03:35,882 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:35,883 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 03:03:35,883 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 03:03:35,883 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY}, enqueuedTicks=862250055364, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 03:03:35,883 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0} 03:03:35,883 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 03:03:35,883 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=862251513091, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 03:03:35,883 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 03:03:35,884 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:03:35,884 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@402a5b2b 03:03:35,884 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 03:03:35,884 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 03:03:35,885 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@223be537 03:03:35,885 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862290707169} 03:03:35,885 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 03:03:35,885 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@305a1158 03:03:35,885 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 03:03:35,885 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 03:03:35,885 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@4e9bbbfc 03:03:35,885 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862290860832} 03:03:35,891 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 03:03:35,891 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:35,891 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:35,891 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 03:03:35,891 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:35,892 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 1 03:03:35,892 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:35,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 03:03:35,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:35,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:35,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 03:03:35,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:35,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 1 03:03:35,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:35,894 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:35,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:35,896 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:03:35,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:03:35,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:35,896 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:35,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 03:03:35,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 03:03:35,896 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 03:03:35,896 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 03:03:35,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 03:03:35,896 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 03:03:35,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2048/0x00000008016a84c0@449d2a69 03:03:35,896 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2048/0x00000008016a84c0@3d69f67b 03:03:35,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:35,896 PM [cluster-test-shard-dispatcher-28] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:35,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f8e09cf}, purged=MutableUnsignedLongSet{size=0}} 03:03:35,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:35,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@292e3337}, purged=MutableUnsignedLongSet{size=0}} 03:03:35,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:35,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:35,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:35,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:35,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:35,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating preCommit 03:03:35,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating preCommit 03:03:35,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 candidate ready 03:03:35,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - candidate rootPath: / 03:03:35,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - messages: [] 03:03:35,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: preCommit - successfulFromPrevious: [] 03:03:35,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 prepared 03:03:35,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:35,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed preCommit 03:03:35,902 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,902 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 candidate ready 03:03:35,902 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 03:03:35,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - messages: [] 03:03:35,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 03:03:35,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 prepared 03:03:35,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:35,903 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed preCommit 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862249234624, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862243170242, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 03:03:35,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 03:03:35,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 03:03:35,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:35,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:35,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 03:03:35,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 03:03:35,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 03:03:35,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 03:03:35,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 03:03:35,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862290860832, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 03:03:35,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1360b95d}, purged=MutableUnsignedLongSet{size=0}} 03:03:35,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@541cfd1f}, purged=MutableUnsignedLongSet{size=0}} 03:03:35,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:35,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:03:35,904 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:35,904 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862290707169, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 03:03:35,904 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:03:35,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 03:03:35,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@2350843f 03:03:35,905 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 03:03:35,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 3 03:03:35,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@2350843f 03:03:35,905 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:35,905 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862311208790} 03:03:35,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 03:03:35,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@62c4b0c7 03:03:35,905 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 03:03:35,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 3 03:03:35,905 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@62c4b0c7 03:03:35,905 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:35,905 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862311368014} 03:03:35,910 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating commit 03:03:35,910 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating commit 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 177 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 03:03:35,911 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 submitted to persistence 03:03:35,911 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 submitted to persistence 03:03:35,912 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:35,912 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=181} 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=177} 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 03:03:35,912 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:35,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 03:03:35,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: commit - successfulFromPrevious: [] 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:03:35,915 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1446501408] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 03:03:35,915 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1134185941] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 03:03:35,916 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:03:35,916 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:03:35,916 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:35,916 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1446501408] 03:03:35,916 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:35,916 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:35,916 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 99.38 μs 03:03:35,916 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-1446501408] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:35,916 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1134185941] 03:03:35,916 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:35,917 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 43.71 μs 03:03:35,917 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-1134185941] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:35,916 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:35,917 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:35,928 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 03:03:35,928 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862311368014, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 03:03:35,928 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} doCommit completed 03:03:35,928 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@12bc782a completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 03:03:35,928 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 03:03:35,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 03:03:35,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:35,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 03:03:35,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 03:03:35,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 03:03:35,928 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 4 03:03:35,929 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@22be388e 03:03:35,929 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#75261711] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#75261711] 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 03:03:35,929 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 4 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@10dd7ba 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:35,929 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:35,929 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 107.7 μs 03:03:35,930 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#75261711] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1808113197] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:03:35,930 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:03:35,931 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1808113197] 03:03:35,931 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1808113197] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862333955368} 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 59.65 μs 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862311208790, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} doCommit completed 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@616a8d9b completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 4 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@320243f3 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 4 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@61ab6775 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:35,937 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862343368533} 03:03:35,938 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf changed state from IDLE to TX_OPEN 03:03:35,939 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:03:35,939 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 03:03:35,939 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:03:35,939 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 0 03:03:35,939 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 1 03:03:35,939 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@5652444d 03:03:35,939 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1}, enqueuedTicks=862345457135} 03:03:35,939 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1}} 03:03:35,940 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 453 bytes, total slices to send: 1 03:03:35,940 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,940 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@24e8954b 03:03:35,940 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=862345804703} 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:35,945 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 03:03:35,946 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 03:03:35,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:35,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:35,963 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5400575 03:03:35,965 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-292: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 03:03:35,967 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-292: Message size: 371 bytes, total slices to send: 1 03:03:35,967 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-292: Message does not need to be sliced - sending original message 03:03:35,974 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 03:03:35,974 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1}, enqueuedTicks=862345457135, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 03:03:35,974 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0} 03:03:35,974 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 03:03:35,974 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=862345804703, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 03:03:35,974 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1} 03:03:35,976 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 1 03:03:35,976 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@36df585a 03:03:35,976 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1}, enqueuedTicks=862382183771} 03:03:35,976 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1}} 03:03:35,976 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 441 bytes, total slices to send: 1 03:03:35,976 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,976 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@ee27b28 03:03:35,976 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=862382488559} 03:03:35,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:35,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:35,978 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3de0962f 03:03:35,978 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-293: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 03:03:35,978 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-293: Message size: 358 bytes, total slices to send: 1 03:03:35,978 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-293: Message does not need to be sliced - sending original message 03:03:35,979 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 03:03:35,979 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1}, enqueuedTicks=862382183771, sessionId=1, txSequence=6} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 03:03:35,979 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0} 03:03:35,981 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 03:03:35,981 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=862382488559, sessionId=1, txSequence=7} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 03:03:35,981 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1} 03:03:35,981 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 2 03:03:35,981 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@11544552 03:03:35,981 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY}, enqueuedTicks=862386814367} 03:03:35,981 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY}} 03:03:35,981 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 290 bytes, total slices to send: 1 03:03:35,981 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,981 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 2 03:03:35,981 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@1b4a26a3 03:03:35,981 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY}, enqueuedTicks=862386970951} 03:03:35,981 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY}} 03:03:35,981 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 291 bytes, total slices to send: 1 03:03:35,981 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,981 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf changed state from TX_OPEN to IDLE 03:03:35,981 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:03:35,981 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf changed state from IDLE to TX_OPEN 03:03:35,981 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:03:35,981 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} allocated sequence 0 03:03:35,981 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@1a982dcb 03:03:35,981 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1, protocol=READY}, enqueuedTicks=862387388722} 03:03:35,981 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1, protocol=READY}} 03:03:35,981 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 427 bytes, total slices to send: 1 03:03:35,981 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,981 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:03:35,982 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf changed state from TX_OPEN to IDLE 03:03:35,982 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:03:35,982 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 3 03:03:35,982 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@5c2a12f4 03:03:35,985 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e skipping transactions [2] 03:03:35,986 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862392351755} 03:03:35,986 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=862392459818} 03:03:35,986 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}} 03:03:35,986 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 290 bytes, total slices to send: 1 03:03:35,986 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,987 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 3 03:03:35,987 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@4b618a64 03:03:35,987 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=862392727855} 03:03:35,987 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}} 03:03:35,987 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 291 bytes, total slices to send: 1 03:03:35,987 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:35,995 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:35,995 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 ready 03:03:35,995 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:03:35,995 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 to ready 03:03:35,997 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipping transactions [[2..2]] 03:03:35,998 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 03:03:35,998 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 03:03:35,998 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.SkipTransactionsPayload, isSendImmediate: false 03:03:35,998 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 is already in state READY (READY) 03:03:35,998 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating canCommit 03:03:35,998 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:35,998 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 validated 03:03:35,998 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed canCommit 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 ready 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 to ready 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 ready 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 to ready 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 is already in state READY (READY) 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating canCommit 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 validated 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:35,999 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed canCommit 03:03:36,0 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY}, enqueuedTicks=862386814367, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=862392459818, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} canCommit completed 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY}, enqueuedTicks=862386970951, sessionId=1, txSequence=8} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1, protocol=READY}, enqueuedTicks=862387388722, sessionId=1, txSequence=9} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=862392727855, sessionId=1, txSequence=10} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 03:03:36,9 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} canCommit completed 03:03:36,9 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 4 03:03:36,9 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@53f7843b 03:03:36,9 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862415446822} 03:03:36,9 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 4 03:03:36,9 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@174c6840 03:03:36,9 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862415509743} 03:03:36,10 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating preCommit 03:03:36,10 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 candidate ready 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - messages: [] 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 prepared 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed preCommit 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating preCommit 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 candidate ready 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - candidate rootPath: / 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - messages: [] 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: preCommit - successfulFromPrevious: [] 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 prepared 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:36,11 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed preCommit 03:03:36,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 03:03:36,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862415446822, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 03:03:36,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} preCommit completed, clearing successfulRequests 03:03:36,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 03:03:36,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862415509743, sessionId=1, txSequence=11} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 03:03:36,12 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} preCommit completed, clearing successfulRequests 03:03:36,12 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 5 03:03:36,12 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@385007d3 03:03:36,12 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862418343254} 03:03:36,12 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 5 03:03:36,12 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@284bb7b7 03:03:36,12 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862418436026} 03:03:36,13 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating commit 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 4, leaderSnapShotIndex: 0 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 submitted to persistence 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating commit 03:03:36,14 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,15 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 225 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 submitted to persistence 03:03:36,15 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 03:03:36,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 03:03:36,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 3 03:03:36,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:36,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 03:03:36,24 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=225} 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 03:03:36,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:36,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 03:03:36,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 03:03:36,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=236} 03:03:36,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 03:03:36,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 4 03:03:36,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:36,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 03:03:36,25 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 03:03:36,26 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 03:03:36,26 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 03:03:36,26 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 03:03:36,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 03:03:36,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 03:03:36,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 03:03:36,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 03:03:36,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 03:03:36,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@1fa30cb7 03:03:36,26 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: commit - successfulFromPrevious: [] 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 03:03:36,27 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 1 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@27979273 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 03:03:36,28 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2068/0x00000008016b6930@2e045f8f 03:03:36,29 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 03:03:36,29 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 03:03:36,29 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 03:03:36,29 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 03:03:36,29 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,29 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,29 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 03:03:36,29 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 03:03:36,29 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:36,30 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 109.1 μs 03:03:36,30 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:03:36,30 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 03:03:36,30 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:03:36,30 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:36,31 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 03:03:36,31 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 82.26 μs 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862333955368, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE}: purge completed 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: purge completed 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@12bc782a purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862418436026, sessionId=1, txSequence=12} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} doCommit completed 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED}: initiating purge 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} allocated sequence 6 03:03:36,32 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@5e8ffdf0 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862438558648} 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862343368533, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@616a8d9b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862392351755, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 03:03:36,33 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e confirmed transaction skip 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 03:03:36,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 03:03:36,34 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862418343254, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 1, lastIndex(): 4 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} doCommit completed 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 6 03:03:36,34 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} allocated sequence 1 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@7e322009 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862439802990} 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,34 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:36,34 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 03:03:36,34 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,35 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:36,35 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 133.3 μs 03:03:36,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 03:03:36,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 03:03:36,35 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 03:03:36,35 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 03:03:36,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:36,35 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@3308560f 03:03:36,35 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=SIMPLE}, enqueuedTicks=862441110112} 03:03:36,35 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=SIMPLE}} 03:03:36,35 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 03:03:36,35 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message size: 291 bytes, total slices to send: 1 03:03:36,35 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-294: Message does not need to be sliced - sending original message 03:03:36,35 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 03:03:36,35 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 03:03:36,35 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 03:03:36,35 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:36,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 4 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:03:36,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}]]] 03:03:36,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 is already in state READY (READY) 03:03:36,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct canCommit 03:03:36,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 validated 03:03:36,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:36,36 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct preCommit 03:03:36,36 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 1.805 ms 03:03:36,37 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 candidate ready 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - candidate rootPath: / 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - messages: [] 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: preCommit - successfulFromPrevious: [] 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 prepared 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct commit 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 5, leaderSnapShotIndex: 2 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 4 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:03:36,38 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 submitted to persistence 03:03:36,39 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 03:03:36,41 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:03:36,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}]]] 03:03:36,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:36,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 03:03:36,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 03:03:36,41 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 03:03:36,42 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 03:03:36,42 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:36,42 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@38a58e4a 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,42 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 03:03:36,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 03:03:36,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 5 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:03:36,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 03:03:36,43 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 03:03:36,43 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 03:03:36,43 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862438558648, sessionId=1, txSequence=13} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 03:03:36,43 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE}: purge completed 03:03:36,44 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=222} 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 5 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 03:03:36,46 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 03:03:36,47 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 5, nextIndex: 6 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: commit - successfulFromPrevious: [] 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 03:03:36,47 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 03:03:36,48 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 03:03:36,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 03:03:36,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=SIMPLE}, enqueuedTicks=862441110112, sessionId=1, txSequence=14} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 03:03:36,48 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:36,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} directCommit completed 03:03:36,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED}: initiating purge 03:03:36,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} allocated sequence 2 03:03:36,48 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@70d56817 03:03:36,49 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:36,49 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862454574448} 03:03:36,49 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 104.6 μs 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 5 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 03:03:36,49 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 03:03:36,49 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 03:03:36,49 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,49 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e invoking destroy 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 03:03:36,49 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 03:03:36,51 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862457104391} 03:03:36,51 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 invoking destroy 03:03:36,51 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862457219155} 03:03:36,51 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf changed state from IDLE to CLOSED 03:03:36,52 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 03:03:36,52 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 358.3 μs 03:03:36,54 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6fffb752: creating a new snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 03:03:36,54 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 03:03:36,54 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 03:03:36,55 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:36,55 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@5999e13d 03:03:36,55 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=862460736272} 03:03:36,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,56 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:36,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 03:03:36,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 03:03:36,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:36,56 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:36,57 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:36,57 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:36,57 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:36,58 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 03:03:36,58 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 03:03:36,58 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:36,58 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-292: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 03:03:36,59 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-292: Message size: 370 bytes, total slices to send: 1 03:03:36,59 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-292: Message does not need to be sliced - sending original message 03:03:36,60 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:36,60 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:36,60 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:36,60 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 03:03:36,60 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 03:03:36,60 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:36,61 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 03:03:36,61 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=862460736272, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 03:03:36,61 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0} 03:03:36,61 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 03:03:36,62 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 03:03:36,62 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:03:36,62 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@6ac6ab9c 03:03:36,62 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=862467766708} 03:03:36,63 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 03:03:36,63 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-293: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 03:03:36,63 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-293: Message size: 232 bytes, total slices to send: 1 03:03:36,63 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-293: Message does not need to be sliced - sending original message 03:03:36,71 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 03:03:36,71 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-567940044], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=862467766708, sessionId=1, txSequence=17} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 03:03:36,71 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0} 03:03:36,71 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:03:36,71 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:36,72 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 03:03:36,72 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 03:03:36,72 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 03:03:36,72 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 03:03:36,72 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70f96497 03:03:36,72 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:36,72 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:36,72 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:03:36,72 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 03:03:36,72 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 03:03:36,73 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@63457ead 03:03:36,74 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:36,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:36,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:36,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:36,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@63457ead 03:03:36,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 03:03:36,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 03:03:36,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:36,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 03:03:36,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:36,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70f96497 03:03:36,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 03:03:36,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 7, leaderSnapShotIndex: 4 03:03:36,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 6 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 03:03:36,75 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 03:03:36,76 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards] 03:03:36,76 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:36,76 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:03:36,76 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 03:03:36,77 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@26f772e6, splicing queue 03:03:36,77 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:03:36,77 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@350a74f9 queue spliced 2 messages 03:03:36,77 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:36,80 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: refreshing backend for shard 2 03:03:36,80 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:03:36,80 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:03:36,80 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:03:36,80 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1789856868] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 3 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:03:36,81 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:03:36,82 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@80a3adc, splicing queue 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@56782601 queue spliced 2 messages 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: refreshing backend for shard 1 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:36,82 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1789856868] 03:03:36,87 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:36,87 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:03:36,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 03:03:36,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 03:03:36,88 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 03:03:36,88 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 6, nextIndex: 7 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@284b4b84 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 6 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2073/0x00000008016bc240@38905da5 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 5 for term 4 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithMultipleShards, matchIndex: 6, lastIndex: 6, isVoting: true 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 03:03:36,89 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=6 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 7, nextIndex: 8 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 5 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 6 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 6 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@474e0c8d 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 7 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 6 to 7 03:03:36,90 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 7 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 03:03:36,103 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2073/0x00000008016bc240@2c8245bc 03:03:36,103 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:36,104 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 6 for term 4 03:03:36,104 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2ceebd5a aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} 03:03:36,104 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 1 03:03:36,105 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@6dd3b761 03:03:36,105 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:36,105 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5a9079b aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} 03:03:36,105 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 1 03:03:36,105 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@5cf47e19 03:03:36,105 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:36,105 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0} 03:03:36,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 03:03:36,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testTransactionChainWithMultipleShards, matchIndex: 7, lastIndex: 7, isVoting: true 03:03:36,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:03:36,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 03:03:36,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leader transfer complete 03:03:36,105 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=7 03:03:36,107 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 6 03:03:36,107 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 03:03:36,107 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 03:03:36,107 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 03:03:36,107 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70} 03:03:36,107 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 03:03:36,108 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 6 03:03:36,108 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 03:03:36,108 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 03:03:36,108 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 03:03:36,108 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 5 for term 4 03:03:36,108 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=6 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 7 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 5, lastIndex(): 7 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 6 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 5 to 6 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 6 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70} 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 7 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 6 to 7 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 7 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 6 for term 4 03:03:36,108 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=7 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} not found in inflight queue, checking pending queue 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862439802990, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6}} 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} not found in inflight queue, checking pending queue 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862457104391, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1}} 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1} 03:03:36,109 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5d620e8e 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} not found in inflight queue, checking pending queue 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862454574448, sessionId=1, txSequence=15} with Envelope{sessionId=1, txSequence=f, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2}} 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE}: purge completed 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} not found in inflight queue, checking pending queue 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044]} with Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-567940044]}, enqueuedTicks=862457219155, sessionId=1, txSequence=16} with Envelope{sessionId=1, txSequence=10, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1}} 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1} 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@66ade5cf: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@3d8db8b3 03:03:36,110 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#269998488], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:03:36,112 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 03:03:36,112 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_35#-1437327921)] 03:03:36,112 PM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 03:03:36,112 PM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_34#-291302607)] 03:03:36,491 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 03:03:36,491 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] ShardManager - shard-manager-testTransactionChainWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1604074124]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:36,492 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 03:03:36,492 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-297: Closing 03:03:36,492 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 03:03:36,493 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 03:03:36,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-298: Closing 03:03:36,493 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 03:03:36,494 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 03:03:36,494 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 03:03:36,494 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 03:03:36,494 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 03:03:36,494 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 03:03:36,495 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 03:03:36,495 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-330966114] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-330966114] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:36,495 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:36,495 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:36,506 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:36,506 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:36,506 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:36,506 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:36,508 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:36,525 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 03:03:36,525 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 03:03:36,525 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 03:03:36,525 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 03:03:36,525 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 03:03:36,525 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 03:03:36,525 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 03:03:36,526 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 03:03:36,527 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:36,527 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:37,517 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7144674574868320386] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:37,545 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:37,545 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:37,546 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:37.548] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:37,548 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3367855382343029589] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:37,563 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:37,563 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:37,564 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:37.565] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:37,587 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:37,593 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3135125926348037450] 03:03:37,593 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:37,596 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:37,596 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:37,596 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:37,596 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:37,605 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:37,605 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:37,605 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:37,609 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:37,617 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4633713230177447633] 03:03:37,617 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:37,618 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:37,619 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:37,630 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:37,630 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:37,636 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:37,644 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8316235687957111390] 03:03:37,644 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:37,645 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:37,646 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:37,647 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:37,647 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:37,660 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:37,661 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:37,662 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 03:03:37,698 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 03:03:37,708 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1026264536] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,715 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:37,722 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:37,728 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 03:03:37,728 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 03:03:37,728 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@15c20202 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7debda02 03:03:37,728 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 03:03:37,728 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@44be013d 03:03:37,728 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 03:03:37,728 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:37,728 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:37,728 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:37,729 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1709823817729, None)) 03:03:37,730 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823817728,0,0)) 03:03:37,730 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7debda02 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@296ed4b3 03:03:37,730 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 03:03:37,731 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:37,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@27360eb6 03:03:37,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:37,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 03:03:37,733 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:37,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:37,733 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378] 03:03:37,734 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:37,734 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 03:03:37,735 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 03:03:37,735 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:37,736 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 03:03:37,737 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,737 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#1636184050 created and ready for shard:member-1-shard-cars-testInstallSnapshot 03:03:37,737 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-344204969] 03:03:37,737 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#1636184050] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,738 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#2097602419] 03:03:37,738 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 03:03:37,738 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 03:03:37,738 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 03:03:37,738 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 03:03:37,739 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 03:03:37,739 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#812775453] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 03:03:37,740 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#812775453] 03:03:37,740 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 03:03:37,740 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 1.013 ms 03:03:37,740 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 1.197 ms: snapshotIndex=5, snapshotTerm=1, journal-size=0 03:03:37,740 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 110.0 μs 03:03:37,741 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:37,741 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 2.331 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 03:03:37,741 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 03:03:37,741 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 03:03:37,741 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 03:03:37,741 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 03:03:37,741 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 03:03:37,742 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 03:03:37,743 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-344204969] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,743 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#812775453] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,749 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 03:03:37,749 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-63407702] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,750 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 03:03:37,761 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 03:03:37,761 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 03:03:37,761 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 03:03:37,761 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 03:03:37,761 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@70c60012 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72fdf226 03:03:37,761 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:37,761 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:37,761 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5ded4a8e 03:03:37,761 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1709823817761, None)) 03:03:37,762 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823817760,0,0)) 03:03:37,762 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@42b37d55 03:03:37,762 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@72fdf226 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6bc2cfee 03:03:37,762 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 03:03:37,763 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:37,763 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 03:03:37,763 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:37,763 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:03:37,763 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-2-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#988547561] 03:03:37,764 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 03:03:37,764 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#988547561] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,764 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1981349822 created and ready for shard:member-2-shard-cars-testInstallSnapshot 03:03:37,764 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#-2065691154] 03:03:37,764 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1981349822] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,764 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#-2065691154] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:37,764 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 03:03:37,768 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:37,768 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:37,768 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 03:03:37,768 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 03:03:37,768 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 03:03:37,768 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 03:03:37,768 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:37,768 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 03:03:37,768 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 03:03:37,768 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 03:03:37,813 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1026264536] 03:03:37,813 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 03:03:37,813 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378] 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 03:03:38,31 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 03:03:38,32 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 03:03:38,32 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 03:03:38,311 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 03:03:38,311 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 03:03:38,311 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 03:03:38,336 PM [cluster-test-shard-dispatcher-16] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:38,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 03:03:38,552 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 03:03:38,552 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 03:03:38,612 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:38,612 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:38,613 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:38,613 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 03:03:38,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:38,613 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:38,613 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:03:38,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:38,613 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:03:38,620 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:38,620 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 03:03:38,620 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:03:38,620 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:38,620 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:03:38,620 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 03:03:38,811 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 03:03:38,811 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 03:03:38,811 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 5 03:03:38,811 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 03:03:38,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:38,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 03:03:38,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 03:03:38,813 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 03:03:38,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=null 03:03:38,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:38,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 03:03:38,813 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 03:03:38,815 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 03:03:38,815 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 03:03:38,816 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 03:03:38,816 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 03:03:38,816 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=13] 03:03:38,817 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:38,817 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:38,817 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 03:03:38,817 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 03:03:38,817 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 03:03:38,817 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 03:03:38,817 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 03:03:38,817 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 03:03:38,817 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 03:03:38,818 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 4 03:03:38,818 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 03:03:38,820 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 03:03:38,820 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@18d9c9bc, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 03:03:38,820 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 03:03:38,820 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 03:03:38,820 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 03:03:38,820 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 03:03:38,820 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 03:03:38,821 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 03:03:38,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 4 03:03:38,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 03:03:38,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 03:03:38,821 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 03:03:38,822 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=5, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 03:03:38,822 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 03:03:38,822 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 03:03:38,822 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@1ff4467a, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 03:03:38,822 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1569958450] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 303.9 μs 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 03:03:38,823 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1569958450] 03:03:38,823 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 03:03:38,823 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 03:03:38,823 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1569958450] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:38,824 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 158.1 μs 03:03:38,824 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 03:03:38,824 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 03:03:38,824 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@286dd7e6: creating a new snapshot member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 03:03:38,824 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 03:03:38,824 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:38,824 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:38,824 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1026264536] 03:03:38,824 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=0} shard 1 03:03:38,825 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 03:03:38,825 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:38,825 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1976244825], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@5ab88be 03:03:38,825 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:03:38,825 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:38,825 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6204f00f] 03:03:38,825 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6204f00f] 03:03:38,825 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1867b50f, attempting to connect 03:03:38,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 03:03:38,825 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 03:03:38,825 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} is not yet known 03:03:38,825 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, nanosAgo=5810, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 03:03:38,825 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:03:38,826 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:03:38,826 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:38,826 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b828598 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@28874044 03:03:38,826 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:03:38,826 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@28874044 03:03:38,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 03:03:38,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 03:03:38,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1976244825], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 03:03:38,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@286dd7e6 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:38,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 03:03:38,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@44fe5bc4 03:03:38,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3ea26e39 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1411427c, splicing queue 03:03:38,828 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3ea26e39 queue spliced 0 messages 03:03:38,829 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.727 ms 03:03:38,829 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0} 03:03:38,829 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 03:03:38,829 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 03:03:38,829 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 03:03:38,829 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 03:03:38,829 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 03:03:38,829 PM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 03:03:38,829 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5fc23fc4 03:03:38,829 PM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:38,829 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5fc23fc4 03:03:38,829 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 03:03:38,830 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@137e9ea6 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@2568d776, splicing queue 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@137e9ea6 queue spliced 0 messages 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: refreshing backend for shard 1 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:38,830 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1026264536] 03:03:38,831 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testInstallSnapshot] 03:03:38,831 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=13] 03:03:38,831 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:38,831 PM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:38,832 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testInstallSnapshot, matchIndex: 5, lastIndex: 5, isVoting: true 03:03:38,832 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Follower's log matches - sending ElectionTimeout 03:03:38,832 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 03:03:38,832 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leader transfer complete 03:03:38,832 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:38,832 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testInstallSnapshot, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 03:03:39,54 PM [cluster-test-shard-dispatcher-11] [WARN] TestShard - member-1-shard-cars-testInstallSnapshot (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-2-shard-cars-testInstallSnapshot, logLastIndex=5, logLastTerm=1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 221, lastApplied : 5, commitIndex : 5 03:03:39,241 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 03:03:39,241 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] ShardManager - shard-manager-testInstallSnapshot: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:39,242 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 03:03:39,242 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-303: Closing 03:03:39,242 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 03:03:39,242 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 03:03:39,243 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 03:03:39,243 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 03:03:39,243 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: shutting down 03:03:39,243 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 03:03:39,243 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 03:03:39,244 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1026264536] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1026264536] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:39,244 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1026264536] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1026264536] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:39,245 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:39,245 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:39,251 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:39,251 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:39,251 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:39,253 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:39,263 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 03:03:39,263 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 03:03:39,263 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 03:03:39,263 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 03:03:39,264 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 03:03:39,265 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:39,265 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:39,680 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:40,260 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3135125926348037450] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:40,277 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:40,278 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:40.281] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:40,284 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4633713230177447633] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:40,297 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:40,298 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:03:40.303] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:40,338 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:40,364 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8879985677167152628] 03:03:40,364 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:40,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:40,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:40,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:40,368 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:40,379 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:40,379 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:40,379 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:40,388 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:40,397 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5435013439390835305] 03:03:40,398 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:40,401 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:40,401 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:40,407 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:40,407 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:40,417 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:40,434 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2742008252986171877] 03:03:40,437 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:40,447 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:40,449 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:40,451 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:40,451 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:40,451 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:40,451 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:40,453 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 03:03:40,473 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1791255545] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:40,474 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:40,476 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 03:03:40,476 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:03:40,476 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5d712cf8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1161216e 03:03:40,476 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6c6677bf 03:03:40,476 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:40,476 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,476 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:40,476 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,477 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709823820477, None)) 03:03:40,477 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823820476,0,0)) 03:03:40,477 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1161216e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@402759a4 03:03:40,477 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 03:03:40,478 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:40,479 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:40,481 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 03:03:40,482 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2cecd48e 03:03:40,482 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:40,482 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,487 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:40,487 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,487 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,487 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] 03:03:40,488 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 03:03:40,488 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:40,488 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-2054494327 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,489 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 03:03:40,489 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-2054494327] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:40,489 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#1673957490] 03:03:40,489 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#1673957490] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:40,489 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:40,489 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:40,490 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:40,490 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 03:03:40,490 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 03:03:40,490 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 03:03:40,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 03:03:40,490 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 03:03:40,497 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 03:03:40,500 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1887001122] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:40,500 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:40,505 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:40,508 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 03:03:40,508 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:03:40,508 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@d2cd8fa to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e2e44c8 03:03:40,508 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:40,508 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3ceb1616 03:03:40,508 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,508 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:40,508 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,509 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709823820508, None)) 03:03:40,511 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823820507,0,0)) 03:03:40,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4954e3f4 03:03:40,513 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7e2e44c8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@692aff52 03:03:40,514 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 03:03:40,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:40,514 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,515 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,515 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#547920568] 03:03:40,516 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 03:03:40,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#547920568] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:40,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1747001546 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,516 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 03:03:40,516 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1747001546] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:40,516 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-236014858] 03:03:40,517 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-236014858] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:40,521 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:40,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:40,527 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:40,527 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:40,527 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:40,527 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 03:03:40,527 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,527 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 03:03:40,527 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 03:03:40,528 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 03:03:40,529 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 03:03:40,532 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 03:03:40,564 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1791255545] 03:03:40,564 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 03:03:40,565 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] 03:03:40,754 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:40,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 03:03:40,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 03:03:40,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:40,755 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 03:03:40,755 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:40,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 03:03:40,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:40,755 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 03:03:40,755 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 03:03:40,756 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 03:03:41,54 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 03:03:41,55 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,55 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 03:03:41,99 PM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:41,314 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 03:03:41,314 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,314 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 03:03:41,385 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:41,385 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:41,386 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:41,387 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,387 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,387 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:41,387 PM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,387 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,387 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,387 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,393 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:41,393 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,393 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,393 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:41,393 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,393 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,393 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,455 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:41,614 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 03:03:41,614 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,614 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 4 03:03:41,614 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,615 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,617 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:41,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:41,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 03:03:41,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:41,618 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:41,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=null 03:03:41,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:41,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,618 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:41,622 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:41,622 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:41,622 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 03:03:41,622 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:41,622 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:41,622 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:41,622 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:03:41,622 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 03:03:41,622 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 03:03:41,623 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 03:03:41,623 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:41,623 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:41,623 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 03:03:41,623 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 03:03:41,623 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:41,624 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:41,625 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:41,627 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 03:03:41,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 03:03:41,628 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,629 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=13] 03:03:41,629 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:41,629 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:41,629 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 03:03:41,629 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 03:03:41,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 03:03:41,629 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:41,631 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#664216717] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:41,632 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:41,633 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:41,634 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,634 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 03:03:41,634 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:03:41,634 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@74cde35e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7d9edf7f 03:03:41,634 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@23c2afad 03:03:41,634 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:41,634 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,634 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1709823821634, None)) 03:03:41,634 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:41,634 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,634 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:41,634 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,634 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823821633,0,0)) 03:03:41,634 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4f4d2b 03:03:41,634 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7d9edf7f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5f42b927 03:03:41,635 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 03:03:41,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:41,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,635 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1298136531] 03:03:41,635 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,635 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,636 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1298136531] 03:03:41,636 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,636 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,636 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1298136531] 03:03:41,636 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 03:03:41,636 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 03:03:41,637 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:03:41,637 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#814628829 created and ready for shard:member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,637 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-3:snapshot-read#-2050483557] 03:03:41,638 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:41,638 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 03:03:41,638 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:41,638 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:41,638 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:41,642 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:41,642 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:41,642 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:41,642 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:41,644 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:41,654 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1791255545] 03:03:41,654 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 03:03:41,654 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] 03:03:41,654 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] 03:03:41,737 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 03:03:42,405 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:03:42,654 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-2742008252986171877] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:42,660 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-2742008252986171877] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:47,505 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 03:03:47,506 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:47,506 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:47,510 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:47,540 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 03:03:47,681 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1887001122] 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 03:03:47,681 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#547920568] 03:03:47,681 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] 03:03:47,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:47,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 03:03:47,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 03:03:47,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 03:03:47,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 03:03:47,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:47,681 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 03:03:47,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 03:03:47,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 03:03:47,682 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 03:03:47,682 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 03:03:47,682 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#547920568] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:47,682 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#547920568] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:47,682 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] 03:03:47,682 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:47,682 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:47,685 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:03:47,685 PM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:03:47,685 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:03:47,685 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:03:47,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:47,688 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 03:03:47,688 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 03:03:47,688 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 03:03:47,688 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 03:03:47,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:47,688 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:47,688 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70e4430f 03:03:47,688 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@70e4430f 03:03:47,688 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 03:03:47,688 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:47,688 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:47,689 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader] 03:03:47,689 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 03:03:47,689 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:47,689 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:47,690 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, matchIndex: -1, lastIndex: -1, isVoting: true 03:03:47,690 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Follower's log matches - sending ElectionTimeout 03:03:47,690 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:47,690 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:47,690 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leader transfer complete 03:03:48,105 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 03:03:48,105 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#983213124]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:48,105 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:48,105 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-308: Closing 03:03:48,105 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 03:03:48,105 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:48,106 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:03:48,106 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 03:03:48,106 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:03:48,106 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:03:48,107 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:48,107 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:48,630 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 03:03:48,630 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=13] 03:03:49,119 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8879985677167152628] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:49,126 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:49,126 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:49,128 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:49.129] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:49,137 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 03:03:49,137 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 03:03:49,137 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 03:03:49,137 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 03:03:49,137 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 03:03:49,138 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:49,138 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:49,140 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:49.141] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:49,167 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:49,177 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1620785854602368311] 03:03:49,177 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:49,178 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:49,178 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:49,189 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:49,189 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:49,196 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:49,196 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:49,196 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:49,198 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b828598 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 03:03:49,198 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:03:49,198 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1976244825], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@7d64905c 03:03:49,198 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 03:03:49,198 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@28874044 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 03:03:49,198 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1976244825]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@3fdfa96e 03:03:49,198 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 03:03:49,198 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#43340378], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:03:49,200 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:49,205 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5198612378081720057] 03:03:49,205 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:49,215 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:49,215 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:49,215 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:49,215 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:49,240 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:49,272 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8360726960275197181] 03:03:49,272 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:49,272 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:49,272 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:49,276 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:49,277 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:49,279 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 03:03:49,283 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:49,283 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:49,293 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 03:03:49,294 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:49,296 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 03:03:49,296 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,296 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:49,296 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,297 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 03:03:49,297 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:03:49,297 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4a042f70 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@226cc972 03:03:49,298 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3a44aab3 03:03:49,298 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1709823829297, None)) 03:03:49,298 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823829296,0,0)) 03:03:49,298 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@226cc972 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7583a016 03:03:49,298 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@626043bd 03:03:49,298 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 03:03:49,299 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:49,301 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:49,303 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 03:03:49,303 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:49,303 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,304 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,304 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,304 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748] 03:03:49,304 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 03:03:49,304 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:49,306 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1792551672 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,306 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1792551672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:49,306 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-1729829947] 03:03:49,306 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 03:03:49,313 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:49,313 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:49,313 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 03:03:49,313 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 03:03:49,313 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:49,313 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 03:03:49,313 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 03:03:49,313 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 03:03:49,313 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 03:03:49,319 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-1729829947] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:49,337 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 03:03:49,338 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-301527011] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:49,341 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:49,344 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:49,350 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:49,354 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 03:03:49,354 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,354 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 03:03:49,354 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:03:49,354 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@27ef69e8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@63bb752c 03:03:49,354 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@16024e73 03:03:49,355 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:49,355 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:49,355 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,355 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1709823829355, None)) 03:03:49,355 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823829354,0,0)) 03:03:49,356 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@63bb752c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@27037ff0 03:03:49,356 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 03:03:49,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@54322843 03:03:49,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:49,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,356 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-974773969] 03:03:49,357 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 03:03:49,357 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 03:03:49,357 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-974773969] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:49,357 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#2098905290 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,358 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1163867001] 03:03:49,358 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#2098905290] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:49,358 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1163867001] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:49,359 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:49,359 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:49,360 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 03:03:49,360 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 03:03:49,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:49,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 03:03:49,360 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,360 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 03:03:49,360 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 03:03:49,360 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 03:03:49,406 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] 03:03:49,406 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 03:03:49,406 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748] 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:03:49,555 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 03:03:49,555 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 03:03:49,556 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 03:03:49,865 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:03:49,865 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:49,865 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 03:03:49,955 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:50,185 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:03:50,185 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,185 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 03:03:50,206 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:50,206 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:50,207 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:50,207 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,207 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,207 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:50,207 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,207 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,207 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,425 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:03:50,425 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,425 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 03:03:50,425 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:50,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 03:03:50,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:50,427 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:50,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 03:03:50,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:50,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,427 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:50,428 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:50,428 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:50,428 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 03:03:50,429 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 03:03:50,429 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 03:03:50,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 03:03:50,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,430 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=13] 03:03:50,430 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:50,430 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:50,430 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 03:03:50,430 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 03:03:50,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 03:03:50,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:50,522 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-301527011] 03:03:50,522 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 03:03:50,522 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-974773969] 03:03:50,522 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,522 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-301527011] 03:03:50,522 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 03:03:50,523 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-974773969] 03:03:50,535 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,535 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,535 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 03:03:50,537 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 03:03:50,537 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:50,537 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-145-0 of 1 operations, client version 13 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-145-0 ready 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-145-0 validated 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-145-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-145-0: success 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,538 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-145-0 candidate ready 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-145-0: canCommit - candidate rootPath: / 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-145-0: canCommit - messages: [] 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-145-0: preCommit - successfulFromPrevious: [] 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-145-0 prepared 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-145-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-145-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,539 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-145-0 submitted to persistence 03:03:50,540 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:50,540 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 03:03:50,540 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:03:50,540 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:50,540 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:50,540 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:03:50,540 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:50,541 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 03:03:50,541 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:50,541 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:03:50,541 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=4483812} is already disabled 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 03:03:50,542 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:03:50,545 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-145-0: commit - successfulFromPrevious: [] 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-145-0 changing state from COMMIT_PENDING to COMMITTED 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-145-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-214#1542246242] 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-560209757] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:50,546 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 03:03:50,546 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-560209757] 03:03:50,547 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 03:03:50,547 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-560209757] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:50,547 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 73.44 μs 03:03:50,547 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21255bbe: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 03:03:50,547 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 03:03:50,547 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:50,547 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:50,547 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] 03:03:50,547 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:50,548 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 03:03:50,548 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7dfe929b] 03:03:50,548 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 03:03:50,548 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:50,548 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1620562358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@303a089 03:03:50,548 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:03:50,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 03:03:50,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:50,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 03:03:50,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 03:03:50,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 03:03:50,548 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-145-0 03:03:50,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1678977534] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 03:03:50,549 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1678977534] 03:03:50,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 03:03:50,549 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 03:03:50,549 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 03:03:50,549 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1678977534] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:50,549 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 128.3 μs 03:03:50,552 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7dfe929b] 03:03:50,552 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@31e57992, attempting to connect 03:03:50,552 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} is not yet known 03:03:50,552 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, nanosAgo=6890, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:03:50,552 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:03:50,552 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@764f6cef instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@12695d32 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@12695d32 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1620562358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21255bbe finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@518c65e9 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5cdc44ac superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@51f973cc, splicing queue 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5cdc44ac queue spliced 0 messages 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 463.5 μs 03:03:50,553 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 03:03:50,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,553 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,554 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 03:03:50,555 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,555 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-146-0 of 1 operations, client version 13 03:03:50,555 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,555 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-146-0 ready 03:03:50,555 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0 canCommit 03:03:50,556 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748)] resolved 03:03:50,556 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0: successfully resolved all cohort actors 03:03:50,556 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0 finishCanCommit 03:03:50,556 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-146-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748)] 03:03:50,556 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748)] 03:03:50,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-146-0 validated 03:03:50,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-146-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:50,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-146-0: success 03:03:50,556 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:03:50,556 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0: canCommit returning result: true 03:03:50,556 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0 commit 03:03:50,556 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0 finish commit 03:03:50,556 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-146-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748)] 03:03:50,556 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748)] 03:03:50,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-146-0 candidate ready 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-146-0: canCommit - candidate rootPath: / 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-146-0: canCommit - messages: [] 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-146-0: preCommit - successfulFromPrevious: [] 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-146-0 prepared 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-146-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-146-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-146-0 submitted to persistence 03:03:50,557 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:50,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 03:03:50,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:03:50,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:50,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 03:03:50,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:50,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 03:03:50,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:50,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:03:50,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 03:03:50,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:03:50,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 03:03:50,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 03:03:50,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 03:03:50,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-146-0: commit - successfulFromPrevious: [] 03:03:50,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-146-0 changing state from COMMIT_PENDING to COMMITTED 03:03:50,559 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-146-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%23-464322748$a] 03:03:50,559 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 03:03:50,559 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:50,559 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 03:03:50,559 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-146-0: commit succeeded 03:03:50,559 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21255bbe: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 03:03:50,559 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 03:03:50,559 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 03:03:50,559 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 92.68 μs 03:03:50,559 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:03:50,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 03:03:50,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:03:50,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 03:03:50,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 03:03:50,560 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 03:03:50,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 03:03:50,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 03:03:50,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 03:03:50,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-146-0 03:03:50,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f4e0409 03:03:50,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@f4e0409 03:03:50,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 03:03:50,560 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 03:03:50,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:50,560 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1e0f0a4e superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@488ac275, splicing queue 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1e0f0a4e queue spliced 0 messages 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:50,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: refreshing backend for shard 1 03:03:50,560 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 52.56 μs 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:50,560 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:50,561 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] 03:03:50,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:50,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 03:03:50,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader] 03:03:50,563 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 03:03:50,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:50,563 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:50,563 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 03:03:50,563 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 03:03:50,563 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:50,563 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leader transfer complete 03:03:50,787 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 217, lastApplied : 2, commitIndex : 2 03:03:50,975 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 03:03:50,975 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:50,976 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,976 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-315: Closing 03:03:50,976 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 03:03:50,976 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 03:03:50,976 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 03:03:50,976 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$ShardNotInitializedTimeout] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:50,976 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 03:03:50,977 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 03:03:50,977 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 03:03:50,977 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:50,977 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#431821912] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:50,977 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 03:03:50,977 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-1620562358] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1620562358] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:50,977 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:50,978 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:50,985 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:50,986 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:50,986 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:50,986 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:50,989 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:03:50,996 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 03:03:50,996 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 03:03:50,996 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 03:03:50,996 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 03:03:50,996 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 03:03:50,996 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:50,997 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:51,300 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:52,0 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1620785854602368311] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:52,10 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5198612378081720057] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:52,17 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:52,19 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:52,20 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:52.021] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:52,27 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:52,27 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:52,30 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:52.031] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:52,47 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:52,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3587436557273250106] 03:03:52,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:52,56 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:52,56 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:52,63 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:52,63 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:52,67 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:52,67 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:52,67 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:52,69 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:52,108 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1772725246398818885] 03:03:52,108 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:52,118 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:52,118 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:52,128 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:52,128 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:52,131 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:52,138 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2724060581087331596] 03:03:52,138 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:52,150 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:52,150 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:52,150 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:52,150 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:52,160 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:52,162 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:52,163 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 03:03:52,176 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 03:03:52,184 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:52,199 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-482687380] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:52,199 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:52,215 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 03:03:52,215 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 03:03:52,215 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:03:52,215 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,215 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5e4db72a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@70b74dbd 03:03:52,215 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:52,215 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2bacf47c 03:03:52,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1709823832215, None)) 03:03:52,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823832214,0,0)) 03:03:52,220 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@f27541e 03:03:52,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@70b74dbd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5efd9254 03:03:52,221 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 03:03:52,221 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:52,223 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:52,224 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 03:03:52,225 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:52,225 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,232 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,232 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,232 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621] 03:03:52,233 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 03:03:52,233 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 03:03:52,234 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 03:03:52,235 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:52,235 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#1496010084 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,235 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#1496010084] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:52,235 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-471533903] 03:03:52,235 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-471533903] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:52,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:52,240 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:52,240 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 03:03:52,240 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 03:03:52,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:52,240 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 03:03:52,240 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 03:03:52,240 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 03:03:52,242 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 03:03:52,242 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1082623122] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:52,252 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 03:03:52,252 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 03:03:52,252 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:03:52,252 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,252 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7568be9f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5700078c 03:03:52,253 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:52,253 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@24c904d4 03:03:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1709823832253, None)) 03:03:52,257 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823832252,0,0)) 03:03:52,258 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5700078c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1bef56d2 03:03:52,258 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@73a2bb4d 03:03:52,258 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 03:03:52,259 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:52,259 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,260 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,260 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,260 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-2-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#-914859509] 03:03:52,260 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 03:03:52,261 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 03:03:52,261 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#-914859509] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:52,261 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1792013634] 03:03:52,261 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#1287936754 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,261 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1792013634] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:52,266 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#1287936754] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:52,272 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:52,272 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:52,272 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:52,272 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:52,272 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 03:03:52,272 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,273 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 03:03:52,273 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 03:03:52,273 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:52,280 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 03:03:52,280 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 03:03:52,280 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 03:03:52,309 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-482687380] 03:03:52,309 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 03:03:52,309 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621] 03:03:52,524 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:52,524 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 03:03:52,524 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 03:03:52,524 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,524 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 03:03:52,524 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:52,525 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 03:03:52,525 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:52,525 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 03:03:52,525 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 03:03:52,525 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 03:03:52,753 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:03:52,754 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:52,754 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 03:03:52,828 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:53,23 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:03:53,23 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,23 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 03:03:53,78 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:53,78 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,78 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:53,78 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,78 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:53,78 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:53,78 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,78 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,78 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,85 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:53,85 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,85 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,85 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:53,85 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,85 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,166 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:53,294 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:03:53,294 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,294 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 03:03:53,294 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,295 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:53,295 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 03:03:53,295 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:53,295 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:53,295 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 03:03:53,295 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:53,295 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,295 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 03:03:53,297 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 03:03:53,297 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 03:03:53,298 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 03:03:53,298 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,298 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:03:53,298 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:53,298 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:53,298 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 03:03:53,298 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 03:03:53,298 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 03:03:53,298 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:53,315 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@64a76ea7: creating a new transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:03:53,315 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 03:03:53,315 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:53,315 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:53,315 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1082623122] 03:03:53,316 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 03:03:53,316 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:03:53,316 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:53,316 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 03:03:53,316 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@510c1120 03:03:53,316 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:03:53,316 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@2db7dbe8 03:03:53,316 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:53,317 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 03:03:53,317 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=13] 03:03:53,317 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4998f86c, attempting to connect 03:03:53,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} is not yet known 03:03:53,319 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=13600, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:03:53,320 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ce8516b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@162c3f0 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@162c3f0 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=3} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:53,320 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:03:53,321 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2088/0x00000008016bf770@50376b2c 03:03:53,321 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:03:53,321 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=3}, enqueuedTicks=879722566122} 03:03:53,321 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=3}} 03:03:53,321 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: Message size: 467 bytes, total slices to send: 1 03:03:53,321 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: Message does not need to be sliced - sending original message 03:03:53,321 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 03:03:53,322 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2089/0x00000008016bf9a8@11351313 03:03:53,322 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=879722566122} 03:03:53,322 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@64a76ea7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:53,322 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 03:03:53,322 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2df2dac8 03:03:53,322 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@64a6309e superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@d979554, splicing queue 03:03:53,322 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@64a6309e queue spliced 0 messages 03:03:53,322 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.949 ms 03:03:53,326 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,327 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-317: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 03:03:53,327 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-317: Message size: 372 bytes, total slices to send: 1 03:03:53,327 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-317: Message does not need to be sliced - sending original message 03:03:53,329 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=3} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 03:03:53,329 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=3}, enqueuedTicks=879722566122, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 03:03:53,329 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=3} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 03:03:53,329 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 03:03:53,329 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=879722566122, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 03:03:53,329 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1} 03:03:53,329 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:03:53,331 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 03:03:53,331 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@47e8ee8a 03:03:53,331 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 03:03:53,332 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 2 03:03:53,332 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2090/0x00000008016c4000@ba3010d 03:03:53,332 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 03:03:53,332 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@257442e8 03:03:53,332 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:53,332 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=1}, enqueuedTicks=879738063478} 03:03:53,332 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=1}} 03:03:53,332 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: Message size: 457 bytes, total slices to send: 1 03:03:53,332 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: Message does not need to be sliced - sending original message 03:03:53,332 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 3 03:03:53,333 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2092/0x00000008016c4470@e32e9e5 03:03:53,333 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=879738849288} 03:03:53,340 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=1} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 03:03:53,340 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=1}, enqueuedTicks=879738063478, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 03:03:53,340 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2} 03:03:53,341 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 03:03:53,341 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=879738849288, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 03:03:53,341 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true} 03:03:53,341 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@64a76ea7 readied transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 03:03:53,341 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} allocated sequence 4 03:03:53,341 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@54ed6c1 03:03:53,341 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:53,341 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=0, protocol=SIMPLE}, enqueuedTicks=879747359531} 03:03:53,341 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=0, protocol=SIMPLE}} 03:03:53,341 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: Message size: 289 bytes, total slices to send: 1 03:03:53,341 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-318: Message does not need to be sliced - sending original message 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transitioned member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 03:03:53,342 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 250 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,343 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 03:03:53,344 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 03:03:53,344 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 03:03:53,349 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 03:03:53,349 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:03:53,349 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:53,349 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=250} 03:03:53,349 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 03:03:53,349 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 03:03:53,349 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:53,349 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7ebe3fba}, purged=MutableUnsignedLongSet{size=0}} 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:03:53,350 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1919938233] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 03:03:53,351 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:53,351 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 03:03:53,351 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1919938233] 03:03:53,351 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 199.2 μs 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#1919938233] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=0, protocol=SIMPLE}, enqueuedTicks=879747359531, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 03:03:53,352 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} directCommit completed 03:03:53,352 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED}: initiating purge 03:03:53,352 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 5 03:03:53,352 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#1139779409]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@3403a27e 03:03:53,352 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:53,352 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#1139779409]}, enqueuedTicks=879758434467} 03:03:53,353 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,353 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@58d92fba}, purged=MutableUnsignedLongSet{size=0}} 03:03:53,353 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:53,353 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,353 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,353 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#490138718] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 03:03:53,353 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 03:03:53,353 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 03:03:53,353 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:53,354 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#490138718] 03:03:53,354 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 03:03:53,354 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:03:53,354 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 57.67 μs 03:03:53,354 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#490138718] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:53,354 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 03:03:53,354 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@64a76ea7: creating a new snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 03:03:53,354 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 03:03:53,354 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:03:53,354 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@124d2a19 03:03:53,354 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=879760437467} 03:03:53,355 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 03:03:53,355 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-317: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 03:03:53,355 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-317: Message size: 396 bytes, total slices to send: 1 03:03:53,355 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-317: Message does not need to be sliced - sending original message 03:03:53,357 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 03:03:53,357 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1139779409], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=879760437467, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 03:03:53,357 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 03:03:53,357 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:03:53,357 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,357 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 03:03:53,358 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 03:03:53,358 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 03:03:53,358 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:03:53,358 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:53,358 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:53,358 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1374e252 03:03:53,358 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1374e252 03:03:53,358 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 03:03:53,358 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:03:53,358 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,358 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 03:03:53,359 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard] 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:03:53,359 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:53,359 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:03:53,359 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@23d4ac34, splicing queue 03:03:53,360 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:03:53,360 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4eca1b48 queue spliced 1 messages 03:03:53,360 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:53,360 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: refreshing backend for shard 1 03:03:53,360 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:03:53,360 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:53,360 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:53,360 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#1082623122] 03:03:53,360 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 03:03:53,360 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:53,360 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: replication of member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@4c3c2820 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leader transfer complete 03:03:53,360 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 03:03:53,361 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} not found in inflight queue, checking pending queue 03:03:53,361 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#1139779409]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} 03:03:53,361 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#1139779409]}, enqueuedTicks=879758434467, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5}} 03:03:53,361 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 03:03:53,361 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@162c3f0 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 03:03:53,361 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 03:03:53,361 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:53,361 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 03:03:53,361 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:03:53,361 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69} 03:03:53,361 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:53,361 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:53,362 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:03:53,362 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 03:03:53,774 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 03:03:53,774 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:53,775 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,775 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-320: Closing 03:03:53,775 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:03:53,775 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 03:03:53,775 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.SendHeartBeat] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:53,775 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 03:03:53,775 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 03:03:53,775 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 03:03:53,775 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 03:03:53,776 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-482687380] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-482687380] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:53,776 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-482687380] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-482687380] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:53,776 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#189523230] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#189523230] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:53,776 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 03:03:53,776 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-482687380] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-482687380] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:53,777 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:53,777 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:03:53.781] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-326)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:53.781] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-326)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:53.781] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-326)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:53.781] [cluster-test-akka.remote.default-remote-dispatcher-9] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-326)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:53,805 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 03:03:53,805 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 03:03:53,805 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 03:03:53,806 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:03:53,806 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 03:03:53,810 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:53,810 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:54,795 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3587436557273250106] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:54,825 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1772725246398818885] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:54,831 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:54,832 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:54,834 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:54.835] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:54,840 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:54,840 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:03:54.843] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:54,873 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:54,881 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8263892596038224686] 03:03:54,881 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:54,882 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:54,882 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:54,885 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:54,885 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:54,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:54,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:54,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:54,894 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:54,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1713812349316016159] 03:03:54,903 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:54,905 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:54,905 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:54,910 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:54,910 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:54,932 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:03:54,942 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1423980765060173441] 03:03:54,942 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:54,944 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:54,944 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:54,957 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@764f6cef aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1620562358], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@9b2b163 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 03:03:54,957 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@12695d32 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1620562358]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@69e8313b 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:03:54,957 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@12695d32 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1620562358]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@2bbe71b1 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-464322748], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:03:54,957 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@162c3f0 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1139779409], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@4c7c318f 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:54,957 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1016532621], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:54,957 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0} 03:03:54,958 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:54,958 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:54,959 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:54,961 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:54,962 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:54,964 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 03:03:54,967 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:54,968 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 03:03:54,968 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:54,977 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 03:03:54,977 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:03:54,978 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5044aa4c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@143b2d28 03:03:54,978 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@66bc66b4 03:03:54,978 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 03:03:54,978 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:54,978 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 03:03:54,978 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:54,978 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:54,978 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:54,979 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1709823834979, None)) 03:03:54,980 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823834978,0,0)) 03:03:54,980 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@143b2d28 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@9a209f8 03:03:54,980 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 03:03:54,981 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:54,983 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:54,985 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 03:03:54,986 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2dbb35df 03:03:54,986 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:54,986 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:54,987 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:54,987 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:54,987 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897] 03:03:54,987 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 03:03:54,987 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:54,987 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:54,987 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267] 03:03:54,987 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 03:03:54,988 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 03:03:54,989 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 03:03:54,989 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 03:03:54,989 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:54,989 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:54,989 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#-161937825 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:54,990 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#-161937825] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:54,994 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 03:03:54,998 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#960456070] 03:03:54,998 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#960456070] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,6 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:55,7 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:55,10 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1801113929] 03:03:55,10 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1801113929] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,11 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#251489780 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,11 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#251489780] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,13 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:55,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:55,13 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:55,13 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:55,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:03:55,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 03:03:55,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 03:03:55,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 03:03:55,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 03:03:55,13 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 03:03:55,13 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 03:03:55,14 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-10910651] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,15 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 03:03:55,15 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:03:55,15 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 03:03:55,15 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@372e11d2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@39e9e947 03:03:55,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,15 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3dd67aa2 03:03:55,15 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,16 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1709823835015, None)) 03:03:55,16 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:55,16 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:55,16 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,16 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823835014,0,0)) 03:03:55,16 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5a7863bc 03:03:55,16 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@39e9e947 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7b3e78db 03:03:55,16 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 03:03:55,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:55,17 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,17 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:03:55,17 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 03:03:55,17 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 03:03:55,17 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 03:03:55,17 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 03:03:55,18 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 03:03:55,18 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 03:03:55,18 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:55,18 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,18 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-2-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2037648592] 03:03:55,18 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,19 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 03:03:55,19 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 03:03:55,19 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 03:03:55,19 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-2037648592] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,19 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#912636880] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,19 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#1715532461 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,19 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-851468915] 03:03:55,19 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#1715532461] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,19 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-1104539975 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,19 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#813286644] 03:03:55,19 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 03:03:55,22 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,22 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-2-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#912636880] 03:03:55,22 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-851468915] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,22 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-1104539975] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,22 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#813286644] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:55,23 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:55,23 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:55,23 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:03:55,23 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 03:03:55,23 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 03:03:55,23 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 03:03:55,23 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,23 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 03:03:55,23 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 03:03:55,23 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 03:03:55,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:55,24 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:55,24 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 03:03:55,24 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 03:03:55,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:03:55,24 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 03:03:55,24 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 03:03:55,24 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:55,24 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 03:03:55,24 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 03:03:55,67 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] 03:03:55,67 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 03:03:55,67 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897] 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:03:55,260 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 03:03:55,261 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 03:03:55,261 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 03:03:55,329 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:55,329 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 03:03:55,329 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 03:03:55,329 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,329 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 03:03:55,329 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:55,329 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 03:03:55,330 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 03:03:55,330 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 03:03:55,330 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 03:03:55,330 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 03:03:55,519 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:03:55,519 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:55,519 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 03:03:55,623 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:55,633 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:55,639 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:03:55,639 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,639 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 03:03:55,799 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:03:55,799 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:55,799 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 03:03:55,869 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:03:55,870 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,870 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 03:03:55,900 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:55,900 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,901 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 03:03:55,959 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:56,89 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:03:56,89 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:56,89 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 03:03:56,90 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:56,91 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:56,91 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 03:03:56,91 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:56,91 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:56,91 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 03:03:56,91 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:56,91 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:56,91 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:56,92 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:56,92 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:56,92 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 03:03:56,92 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:56,92 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:56,92 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 03:03:56,92 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 03:03:56,92 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 03:03:56,93 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 03:03:56,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:56,93 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:56,93 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 03:03:56,93 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 03:03:56,93 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:56,93 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 03:03:56,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:56,93 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:56,93 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 03:03:56,94 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 03:03:56,94 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:56,109 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 03:03:56,109 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:03:56,109 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 03:03:56,109 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 03:03:56,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:56,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 03:03:56,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:56,110 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:56,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 03:03:56,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:56,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 03:03:56,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 03:03:56,111 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 03:03:56,111 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 03:03:56,111 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 03:03:56,112 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 03:03:56,112 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 03:03:56,112 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 03:03:56,112 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=13] 03:03:56,112 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 03:03:56,112 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 03:03:56,112 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 03:03:56,112 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 03:03:56,113 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:56,174 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] 03:03:56,174 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 03:03:56,174 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267] 03:03:56,174 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7e64956: creating a new transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,176 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 03:03:56,176 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:56,176 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:56,176 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] 03:03:56,176 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:56,176 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 1 03:03:56,176 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@30d2f5c2] 03:03:56,176 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,176 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 0 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 03:03:56,176 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 03:03:56,176 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 03:03:56,176 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] 03:03:56,177 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 03:03:56,177 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 0 03:03:56,177 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@12aac551] 03:03:56,177 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,177 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@30d2f5c2] 03:03:56,177 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@12aac551] 03:03:56,177 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5cc92852, attempting to connect 03:03:56,177 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} allocated sequence 0 03:03:56,177 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:56,177 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard default resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@39048ed3, attempting to connect 03:03:56,177 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7e64956 readied transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,177 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@4a817313 03:03:56,177 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 03:03:56,177 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} scheduling timeout in 30 seconds 03:03:56,178 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=9960, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:03:56,178 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@4a00d199 03:03:56,178 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:03:56,178 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 03:03:56,178 PM [ForkJoinPool.commonPool-worker-1] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:03:56,178 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=5940, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:03:56,178 PM [ForkJoinPool.commonPool-worker-1] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:03:56,178 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:03:56,178 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:56,178 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50173848 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1515bc2e 03:03:56,178 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:03:56,178 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1515bc2e 03:03:56,180 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,180 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 03:03:56,180 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 03:03:56,180 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], modifications=1, protocol=THREE_PHASE} 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 0 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@4a00d199 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f7098a7 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], coordinated=true}, enqueuedTicks=882583751337} 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7e64956 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@604850ba 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1cf702a superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3d87fc6a, splicing queue 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1cf702a queue spliced 0 messages 03:03:56,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 4.241 ms 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 0 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present} 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@609fedd instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2466a326 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} previous state FLUSHED 03:03:56,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating canCommit 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2466a326 03:03:56,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 03:03:56,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 validated 03:03:56,182 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], modifications=1, protocol=THREE_PHASE} 03:03:56,182 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 0 03:03:56,183 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed canCommit 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@4a817313 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5c341abc transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], coordinated=true}, enqueuedTicks=882583630794} 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7e64956 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1502f8f1 03:03:56,183 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@46c06e4 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4cbb4000, splicing queue 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@46c06e4 queue spliced 0 messages 03:03:56,183 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating canCommit 03:03:56,183 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,183 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 03:03:56,183 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:56,183 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed canCommit 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} in 441.2 μs 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], coordinated=true} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], coordinated=true}, enqueuedTicks=882583751337, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], coordinated=true} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085], coordinated=true}, enqueuedTicks=882583630794, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 03:03:56,183 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} canCommit completed 03:03:56,183 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 1 03:03:56,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@9486dd9 03:03:56,184 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 03:03:56,184 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} 03:03:56,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 1 03:03:56,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@9486dd9 03:03:56,184 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 03:03:56,184 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5c341abc transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882589800798} 03:03:56,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:03:56,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@eee75f3 03:03:56,184 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating preCommit 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,184 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} 03:03:56,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 1 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 03:03:56,184 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@eee75f3 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 03:03:56,184 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:56,184 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f7098a7 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882590029224} 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:56,184 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed preCommit 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating preCommit 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 candidate ready 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - messages: [] 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 prepared 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:56,184 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed preCommit 03:03:56,189 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 03:03:56,189 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882589800798, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 03:03:56,189 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:03:56,189 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 03:03:56,189 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882590029224, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 03:03:56,189 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:03:56,189 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 2 03:03:56,189 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@4f3686d6 03:03:56,189 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 03:03:56,189 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} 03:03:56,189 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 2 03:03:56,189 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@4f3686d6 03:03:56,189 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 03:03:56,189 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5c341abc transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882595411428} 03:03:56,189 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:03:56,189 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@3760ab0 03:03:56,189 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 03:03:56,189 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} 03:03:56,189 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating commit 03:03:56,189 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 2 03:03:56,189 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,189 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@3760ab0 03:03:56,189 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:56,190 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f7098a7 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882595630404} 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating commit 03:03:56,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 03:03:56,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 03:03:56,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:56,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:56,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 03:03:56,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 03:03:56,190 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 03:03:56,190 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 submitted to persistence 03:03:56,191 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 03:03:56,191 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 03:03:56,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 03:03:56,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:03:56,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:56,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 03:03:56,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 03:03:56,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:56,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 03:03:56,192 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 03:03:56,192 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 03:03:56,192 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 03:03:56,193 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6c198d9f}, purged=MutableUnsignedLongSet{size=0}} 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@26af0d37}, purged=MutableUnsignedLongSet{size=0}} 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#361073415] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#2013247050] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] and replicatedToAllIndex is -1 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:03:56,193 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 03:03:56,193 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 03:03:56,194 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 03:03:56,194 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:03:56,194 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 03:03:56,194 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:03:56,194 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#361073415] 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882595411428, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} doCommit completed 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@609fedd completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: initiating purge 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#2013247050] 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 3 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@30a642d1 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED}: initiating purge 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} allocated sequence 3 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@32e1f144 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5c341abc transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882600340411} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 71.46 μs 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 81.41 μs 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882595630404, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#361073415] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#2013247050] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:56,194 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50173848 completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 03:03:56,195 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7e64956: creating a new snapshot member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@565e9ada 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED}: initiating purge 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} allocated sequence 3 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@40f36b51 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:56,195 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f7098a7 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882600693040} 03:03:56,195 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:03:56,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:03:56,195 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@e6ac0d8, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:03:56,195 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 03:03:56,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 03:03:56,195 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 03:03:56,195 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@b1286bf, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:03:56,195 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 03:03:56,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,195 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 03:03:56,195 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 03:03:56,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 03:03:56,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 03:03:56,195 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,195 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 03:03:56,195 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 03:03:56,195 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:56,195 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 03:03:56,196 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 03:03:56,196 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 03:03:56,196 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 03:03:56,196 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 03:03:56,196 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 03:03:56,196 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5f4a7eee}, purged=MutableUnsignedLongSet{size=0}} 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-67290336] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] and replicatedToAllIndex is -1 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 03:03:56,197 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 03:03:56,198 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,198 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1c8120ba}, purged=MutableUnsignedLongSet{size=0}} 03:03:56,198 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 03:03:56,198 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,198 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1953665890] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 03:03:56,198 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-67290336] 03:03:56,198 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:03:56,198 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-67290336] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:56,198 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1953665890] 03:03:56,199 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 22.19 μs 03:03:56,199 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 03:03:56,199 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1953665890] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:56,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:03:56,199 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 79.50 μs 03:03:56,199 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@42991029, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:03:56,199 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 03:03:56,199 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 03:03:56,196 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ecd5c23 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@732a6e3b 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@732a6e3b 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1ecd5c23 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 03:03:56,200 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 03:03:56,200 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 03:03:56,200 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:56,200 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 03:03:56,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 3 03:03:56,200 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5c341abc superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@42411954, splicing queue 03:03:56,200 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5c341abc queue spliced 1 messages 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 0 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present} 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f7098a7 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@5dbd76eb, splicing queue 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3f7098a7 queue spliced 1 messages 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 1 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:56,201 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:56,201 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 03:03:56,202 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 03:03:56,202 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@6431a938, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 3 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 03:03:56,203 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 03:03:56,206 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite] 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite] 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=13] 03:03:56,207 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 03:03:56,207 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 03:03:56,207 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 03:03:56,207 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@799196dd 03:03:56,207 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 03:03:56,207 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 03:03:56,207 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 03:03:56,207 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,207 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@53d03aa1 03:03:56,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 03:03:56,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 03:03:56,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:56,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} 03:03:56,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSnapshotOnRootOverwrite, matchIndex: 1, lastIndex: 1, isVoting: true 03:03:56,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882600693040, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3}} 03:03:56,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 03:03:56,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE}: purge completed 03:03:56,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:56,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1515bc2e purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} 03:03:56,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:03:56,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 03:03:56,208 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@50173848 purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 03:03:56,208 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 03:03:56,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:56,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-default-testSnapshotOnRootOverwrite, matchIndex: 1, lastIndex: 1, isVoting: true 03:03:56,208 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Follower's log matches - sending ElectionTimeout 03:03:56,209 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:56,209 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leader transfer complete 03:03:56,209 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 03:03:56,209 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 03:03:56,209 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:56,209 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 03:03:56,209 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 03:03:56,209 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57} 03:03:56,209 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 03:03:56,209 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 03:03:56,209 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 03:03:56,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} not found in inflight queue, checking pending queue 03:03:56,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 03:03:56,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1607555085]}, enqueuedTicks=882600340411, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 03:03:56,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE}: purge completed 03:03:56,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2466a326 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} 03:03:56,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: purge completed 03:03:56,209 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@609fedd purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 03:03:56,210 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 03:03:56,210 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:56,210 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 03:03:56,210 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 03:03:56,210 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 03:03:56,210 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 03:03:56,210 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 03:03:56,619 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 03:03:56,619 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] ShardManager - shard-manager-testSnapshotOnRootOverwrite: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:03:56,619 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 03:03:56,619 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 03:03:56,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-328: Closing 03:03:56,619 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-327: Closing 03:03:56,619 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 03:03:56,619 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 03:03:56,620 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 03:03:56,620 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$ShardNotInitializedTimeout] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-2117833278] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:56,620 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 03:03:56,620 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 03:03:56,621 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 03:03:56,621 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 03:03:56,622 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:56,622 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:56,626 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:56,626 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:56,626 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:56,626 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:56,644 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 03:03:56,644 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 03:03:56,644 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 03:03:56,644 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 03:03:56,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 03:03:56,644 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 03:03:56,644 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 03:03:56,644 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 03:03:56,645 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:56,645 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:57,638 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-8263892596038224686] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:57,659 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1713812349316016159] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:03:57,664 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:57,664 PM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:03:57.666] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:57,675 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:57,675 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:57,677 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:03:57.678] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:57,698 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:57,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7771105880773882563] 03:03:57,704 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:03:57,705 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:03:57,705 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:03:57,710 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:57,710 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:57,715 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:03:57,715 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:03:57,715 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:03:57,719 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:03:57,727 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5632809588833605017] 03:03:57,727 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:03:57,729 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:57,729 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:03:57,732 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:57,732 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:57,747 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:03:57,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4339179919468037359] 03:03:57,757 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:03:57,759 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:03:57,759 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:03:57,767 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:57,768 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:57,770 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:03:57,770 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:03:57,770 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 03:03:57,775 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2047373354] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:57,786 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:03:57,790 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:57,794 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:03:57,796 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:03:57,798 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 03:03:57,817 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 03:03:57,817 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:03:57,817 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@76fb0927 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@189d3801 03:03:57,817 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@9783d81 03:03:57,819 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709823837818, None)) 03:03:57,819 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 03:03:57,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,819 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:57,819 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,819 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823837817,0,0)) 03:03:57,819 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2426d04b 03:03:57,819 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@189d3801 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6dffa4b3 03:03:57,819 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 03:03:57,820 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:57,821 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:03:57,822 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:57,822 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,823 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,823 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,823 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167] 03:03:57,823 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 03:03:57,823 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 03:03:57,823 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:57,823 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-301035159 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,823 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 03:03:57,824 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-301035159] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:57,826 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1278197877] 03:03:57,826 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1278197877] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:57,828 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 03:03:57,828 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#912229953] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:57,829 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:57,829 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:57,829 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:57,829 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:03:57,829 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 03:03:57,829 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 03:03:57,832 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 03:03:57,832 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 03:03:57,836 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 03:03:57,836 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 03:03:57,836 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:03:57,836 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@63417e8f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@77d38092 03:03:57,836 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1b27c61d 03:03:57,836 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709823837836, None)) 03:03:57,836 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 03:03:57,837 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:57,837 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823837835,0,0)) 03:03:57,837 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@77d38092 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5c54f8d9 03:03:57,837 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 03:03:57,837 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:03:57,837 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,837 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@38aa20a6 03:03:57,846 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:03:57,846 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:57,846 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,846 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:57,846 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1266585430] 03:03:57,847 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 03:03:57,847 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#1266585430] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:57,847 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#2080806483 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:57,847 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#584322915] 03:03:57,847 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#2080806483] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:57,847 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#584322915] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:57,848 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 03:03:57,849 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:03:57,849 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:03:57,849 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 03:03:57,849 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 03:03:57,849 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:57,849 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:03:57,849 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:57,849 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 03:03:57,849 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 03:03:57,849 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 03:03:57,887 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2047373354] 03:03:57,887 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 03:03:57,887 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167] 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:03:58,54 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:03:58,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 03:03:58,55 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 03:03:58,293 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:03:58,293 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,294 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 03:03:58,397 PM [cluster-test-shard-dispatcher-18] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:03:58,603 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:03:58,603 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,603 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 03:03:58,724 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:03:58,724 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:03:58,725 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:58,725 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 03:03:58,725 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,725 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:58,725 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,725 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,725 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,729 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:03:58,729 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 03:03:58,729 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,730 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:03:58,730 PM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,730 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,783 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:03:58,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:03:58,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,864 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 03:03:58,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,865 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:03:58,865 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 03:03:58,865 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:03:58,865 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:03:58,865 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 03:03:58,866 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:03:58,866 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,866 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:03:58,874 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:03:58,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:03:58,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 03:03:58,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:58,875 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:03:58,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:03:58,875 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 03:03:58,876 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 03:03:58,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:03:58,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:58,876 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:58,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 03:03:58,876 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 03:03:58,880 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 03:03:58,880 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,880 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:03:58,880 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:03:58,880 PM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:03:58,880 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 03:03:58,880 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 03:03:58,880 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 03:03:58,880 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:03:58,893 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1befd53: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:03:58,893 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 03:03:58,893 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:58,893 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:58,893 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#912229953] 03:03:58,893 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 03:03:58,893 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:03:58,893 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:58,894 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1befd53 readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:03:58,894 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@767014a7 03:03:58,894 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:03:58,894 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:58,894 PM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 03:03:58,894 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=13] 03:03:58,894 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@c4ecf3d, attempting to connect 03:03:58,896 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 03:03:58,896 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=11110, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:03:58,896 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7e8887d4 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@78a300ae 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@78a300ae 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=4, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2102/0x00000008016c6a50@7d378fa0 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=4, protocol=SIMPLE}, enqueuedTicks=885299926875} 03:03:58,897 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=4, protocol=SIMPLE}} 03:03:58,898 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message size: 510 bytes, total slices to send: 1 03:03:58,898 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message does not need to be sliced - sending original message 03:03:58,898 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1befd53 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:58,898 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 03:03:58,898 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@37540d64 03:03:58,898 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1a79219d superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@191c3438, splicing queue 03:03:58,898 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1a79219d queue spliced 0 messages 03:03:58,898 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.175 ms 03:03:58,900 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 03:03:58,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 246 03:03:58,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 03:03:58,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 03:03:58,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:58,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:58,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 03:03:58,902 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 03:03:58,903 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=246} 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 03:03:58,903 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@785cd6d3}, purged=MutableUnsignedLongSet{size=0}} 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:03:58,905 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-2144336788] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 03:03:58,905 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=4, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 03:03:58,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=4, protocol=SIMPLE}, enqueuedTicks=885299926875, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=4, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@5890d07 03:03:58,906 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED}: initiating purge 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 1 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@12a10fff 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358]}, enqueuedTicks=885311769400} 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-2144336788] 03:03:58,906 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1befd53: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:03:58,906 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 03:03:58,906 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:03:58,906 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@2508556f 03:03:58,906 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=885312174520} 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 107.8 μs 03:03:58,906 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-2144336788] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 03:03:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 03:03:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@18ec1cb}, purged=MutableUnsignedLongSet{size=0}} 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,907 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 03:03:58,907 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#867609549] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 03:03:58,907 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#867609549] 03:03:58,907 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:03:58,908 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 03:03:58,908 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 231.4 μs 03:03:58,908 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#867609549] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:03:58,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}]]] 03:03:58,912 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 03:03:58,912 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-330: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 03:03:58,913 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-330: Message size: 392 bytes, total slices to send: 1 03:03:58,913 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-330: Message does not need to be sliced - sending original message 03:03:58,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}]]] 03:03:58,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:03:58,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:03:58,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 03:03:58,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 03:03:58,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 03:03:58,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:58,913 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 03:03:58,914 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 03:03:58,914 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=885312174520, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 03:03:58,914 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 03:03:58,914 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2bb4511: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: replication of member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@5fe10206 03:03:58,914 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 03:03:58,914 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:03:58,914 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,914 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2047373354] 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:03:58,914 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 03:03:58,914 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 03:03:58,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:03:58,914 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:03:58,914 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-111405229], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@74196470 03:03:58,915 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 03:03:58,915 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:03:58,915 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:03:58,915 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6c2694e3] 03:03:58,915 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6c2694e3] 03:03:58,915 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 03:03:58,915 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358]}, enqueuedTicks=885311769400, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 03:03:58,915 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 03:03:58,915 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@78a300ae purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 03:03:58,915 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:03:58,915 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7e8887d4 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 03:03:58,915 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@39793293, attempting to connect 03:03:58,915 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 1 03:03:58,915 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=5010, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:03:58,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 03:03:58,916 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4901e2a1 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@47c615f1 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@47c615f1 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-111405229], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2bb4511 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6fb1fca 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@40a9f15e superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@37371d27, splicing queue 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@40a9f15e queue spliced 0 messages 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 294.3 μs 03:03:58,916 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 03:03:58,916 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1befd53: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:03:58,916 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:03:58,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=OPEN} allocated sequence 0 03:03:58,916 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1befd53 readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 03:03:58,916 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@2ba9378 03:03:58,916 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:58,916 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=1, protocol=SIMPLE}, enqueuedTicks=885322435136} 03:03:58,916 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=1, protocol=SIMPLE}} 03:03:58,916 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message size: 418 bytes, total slices to send: 1 03:03:58,916 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message does not need to be sliced - sending original message 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 ready 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 to ready 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct canCommit 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 validated 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct preCommit 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 candidate ready 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - candidate rootPath: / 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - messages: [] 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 prepared 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct commit 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 213 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 03:03:58,917 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 03:03:58,918 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:03:58,918 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:03:58,918 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,918 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 03:03:58,918 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 submitted to persistence 03:03:58,918 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 03:03:58,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 03:03:58,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 03:03:58,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:03:58,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=213} 03:03:58,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 03:03:58,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 03:03:58,918 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:03:58,919 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: commit - successfulFromPrevious: [] 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:03:58,919 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 03:03:58,919 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:03:58,919 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 79.81 μs 03:03:58,919 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=1, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 03:03:58,920 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=1, protocol=SIMPLE}, enqueuedTicks=885322435136, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 03:03:58,920 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} directCommit completed 03:03:58,920 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED}: initiating purge 03:03:58,920 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} allocated sequence 1 03:03:58,920 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@262070eb 03:03:58,920 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1befd53: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 03:03:58,920 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:03:58,920 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358]}, enqueuedTicks=885325744509} 03:03:58,920 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 03:03:58,920 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 0 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 03:03:58,920 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2090/0x00000008016c4000@501a89d1 03:03:58,920 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=885325985585} 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 03:03:58,920 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 03:03:58,920 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 03:03:58,921 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 60.87 μs 03:03:58,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:03:58,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 03:03:58,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 03:03:58,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:03:58,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 03:03:58,922 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 03:03:58,922 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 03:03:58,923 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=885325985585, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 03:03:58,923 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true} 03:03:58,923 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1befd53: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 03:03:58,923 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 03:03:58,923 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 0 03:03:58,923 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@3b149aff 03:03:58,923 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=885328953269} 03:03:58,923 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 03:03:58,924 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-330: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 03:03:58,924 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-330: Message size: 369 bytes, total slices to send: 1 03:03:58,924 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-330: Message does not need to be sliced - sending original message 03:03:58,924 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 03:03:58,924 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=885328953269, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 03:03:58,924 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0} 03:03:58,924 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2bb4511: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 03:03:58,924 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 03:03:58,925 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-2047373354] 03:03:58,925 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 03:03:58,925 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167] 03:03:58,925 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167] 03:03:58,976 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:03:58,976 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 03:03:58,976 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-333: Closing 03:03:58,976 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:03:58,976 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 03:03:58,977 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:58,977 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:58,982 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:58,982 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:58,983 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:58,983 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:03:58,997 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:03:58,998 PM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:03:58,998 PM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 03:03:58,998 PM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:03:58,998 PM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 03:03:58,998 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:03:58,998 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:03:59,993 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7771105880773882563] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:00,13 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5632809588833605017] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:00,20 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:00,20 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:00,21 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:00.022] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:00,43 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:00,53 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [-5096530505976530292] 03:04:00,53 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:00,57 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:00,57 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 03:04:00,58 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:00,58 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:00,69 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:00,70 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:00,73 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 03:04:00,80 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1268445906] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,80 PM [reinstated-member2-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 03:04:00,83 PM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:04:00,84 PM [reinstated-member2-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 03:04:00,84 PM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 03:04:00,84 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 03:04:00,84 PM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@dac614 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6a4c7ddd 03:04:00,84 PM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@17f784c3 03:04:00,84 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 03:04:00,84 PM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1709823840084, None)) 03:04:00,84 PM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823840083,0,0)) 03:04:00,84 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@27fdf70a 03:04:00,84 PM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6a4c7ddd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5a3d9b 03:04:00,85 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 03:04:00,85 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:00,85 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:04:00,85 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 03:04:00,86 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 03:04:00,86 PM [reinstated-member2-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,86 PM [reinstated-member2-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-647203851 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 03:04:00,86 PM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-170650440] 03:04:00,86 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#-647203851] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,86 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-170650440] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,86 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 03:04:00,87 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 03:04:00,87 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#-1956441401] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,87 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#427823900 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 03:04:00,87 PM [reinstated-member2-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-2147460987] 03:04:00,87 PM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#427823900] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,87 PM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 246 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 03:04:00,90 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 03:04:00,91 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5959ef35}, purged=MutableUnsignedLongSet{size=0}} 03:04:00,91 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:04:00,91 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:04:00,91 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 03:04:00,91 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 03:04:00,91 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 65 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 1 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 213 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 2 to 2 03:04:00,92 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 2.634 ms - Switching actor to Follower - last log index = 2, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 3 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:04:00,93 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 03:04:00,93 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 03:04:00,93 PM [reinstated-member2-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:00,93 PM [reinstated-member2-shard-dispatcher-12] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 03:04:00,93 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 03:04:00,93 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:04:00,94 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 03:04:00,94 PM [reinstated-member2-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:04:00,94 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:04:00,94 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 03:04:00,94 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 03:04:00,94 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 03:04:00,94 PM [reinstated-member2-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:04:00,94 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:00,94 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:00,94 PM [reinstated-member2-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 03:04:00,95 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-2147460987] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,96 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 03:04:00,96 PM [reinstated-member2-shard-dispatcher-19] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:00,96 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 03:04:00,96 PM [reinstated-member2-shard-dispatcher-18] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 03:04:00,96 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:00,96 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 03:04:00,96 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 03:04:00,97 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 03:04:00,97 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 03:04:00,97 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 03:04:00,97 PM [reinstated-member2-shard-dispatcher-19] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 03:04:00,97 PM [reinstated-member2-shard-dispatcher-19] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:00,97 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 03:04:00,97 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:00,97 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 03:04:00,97 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 03:04:00,98 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 03:04:00,98 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 03:04:00,98 PM [reinstated-member2-shard-dispatcher-19] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 03:04:00,98 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:04:00,98 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 03:04:00,98 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 03:04:00,98 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 03:04:00,98 PM [reinstated-member2-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=13] 03:04:00,98 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:00,99 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:00,99 PM [reinstated-member2-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 03:04:00,99 PM [reinstated-member2-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 03:04:00,135 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1268445906] 03:04:00,136 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 03:04:00,136 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797] 03:04:00,136 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12beb4bf: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 03:04:00,136 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 03:04:00,136 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:00,136 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:00,136 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1268445906] 03:04:00,136 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:00,137 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@741a9c1f] 03:04:00,137 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=0} shard 1 03:04:00,137 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 03:04:00,137 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:00,137 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-456265427], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@231687c6 03:04:00,137 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} scheduling timeout in 30 seconds 03:04:00,137 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@741a9c1f] 03:04:00,137 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@419aecae, attempting to connect 03:04:00,137 PM [reinstated-member2-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: retiring state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=43741901, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 03:04:00,138 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, nanosAgo=9300, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 03:04:00,138 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4e3fa4ac instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6557d60c 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6557d60c 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-456265427], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12beb4bf finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7f3baa27 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1db9863c superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2e105be3, splicing queue 03:04:00,138 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1db9863c queue spliced 0 messages 03:04:00,139 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 550.4 μs 03:04:00,139 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0} 03:04:00,139 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 03:04:00,139 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:04:00,139 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 03:04:00,139 PM [reinstated-member2-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 03:04:00,139 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 03:04:00,139 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@dcfe889 03:04:00,139 PM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@dcfe889 03:04:00,139 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 03:04:00,139 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@55ce4be3 03:04:00,140 PM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@55ce4be3 03:04:00,140 PM [reinstated-member2-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 03:04:00,140 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-337: Closing 03:04:00,140 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:04:00,140 PM [reinstated-member2-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 03:04:00,140 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-338: Closing 03:04:00,140 PM [reinstated-member2-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 03:04:00,140 PM [reinstated-member2-akka.actor.default-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 03:04:00,140 PM [reinstated-member2-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 03:04:00,141 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:04:00,141 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:04:00,141 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:00,141 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:04:00,141 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 03:04:00,141 PM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:00,142 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:04:00,142 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 03:04:00,142 PM [reinstated-member2-akka.actor.default-dispatcher-14] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 03:04:00,143 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1268445906] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1268445906] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,143 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1268445906] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#1268445906] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,143 PM [reinstated-member2-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://reinstated-member2/user/$a#-456265427] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#-456265427] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,143 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:00,143 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:00,145 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:00,154 PM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:00,155 PM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:04:00.157] [reinstated-member2-akka.actor.internal-dispatcher-3] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:00,170 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:00,178 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7771689472981444931] 03:04:00,178 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:00,180 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:00,180 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:00,191 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:00,191 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:00,191 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:00,191 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:00,191 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:00,195 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:00,202 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2846220367935070154] 03:04:00,202 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:00,204 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:00,204 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:00,209 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:00,209 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:00,218 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:04:00,224 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [9158954237680329073] 03:04:00,224 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:00,225 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:00,225 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:00,237 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2466a326 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} 03:04:00,237 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@50d9b8ca 03:04:00,237 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 03:04:00,237 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#139583267], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 03:04:00,237 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1515bc2e aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, state=OPEN} 03:04:00,237 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1607555085]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@78903a03 03:04:00,237 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:00,237 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#270272897], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:00,237 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0} 03:04:00,237 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@78a300ae aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 03:04:00,237 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 03:04:00,237 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@20fbff59 03:04:00,237 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT}, enqueuedTicks=886643631878} 03:04:00,238 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT}} 03:04:00,238 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message size: 287 bytes, total slices to send: 1 03:04:00,238 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message does not need to be sliced - sending original message 03:04:00,238 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0} 03:04:00,238 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4901e2a1 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-111405229], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@1c5f08df 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 03:04:00,238 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@47c615f1 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-111405229]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@1be46b3b 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:00,238 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@78a300ae aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 1 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@12756f36 03:04:00,238 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT}, enqueuedTicks=886644133000} 03:04:00,238 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT}} 03:04:00,238 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message size: 287 bytes, total slices to send: 1 03:04:00,238 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message does not need to be sliced - sending original message 03:04:00,238 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0} 03:04:00,238 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@78a300ae aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 1 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@219080d0 03:04:00,238 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@47bb01f5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT}, enqueuedTicks=886644315095} 03:04:00,238 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1969055358], modifications=0, protocol=ABORT}} 03:04:00,238 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message size: 287 bytes, total slices to send: 1 03:04:00,238 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-331: Message does not need to be sliced - sending original message 03:04:00,238 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0} 03:04:00,238 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@47c615f1 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-111405229]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@1061e5e5 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#1831578167], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:00,238 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4e3fa4ac aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=1, replyTo=Actor[akka://reinstated-member2/user/$a#-456265427], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@4241427c 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 03:04:00,238 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@6557d60c aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 03:04:00,238 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#-456265427]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@296986b0 03:04:00,239 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1025033797], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:00,241 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:00,242 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:00,242 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:00,242 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:00,243 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,265 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,266 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1334993715] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,274 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 03:04:00,274 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:04:00,274 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@586f9b8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2f75d440 03:04:00,274 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@34c91dc6 03:04:00,274 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,274 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,274 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:00,275 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,275 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1709823840275, None)) 03:04:00,275 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823840274,0,0)) 03:04:00,275 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2f75d440 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@53fc6e1b 03:04:00,276 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 03:04:00,277 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:00,279 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:00,281 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@55ca7e31 03:04:00,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:00,286 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,291 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,291 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,291 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416] 03:04:00,292 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 03:04:00,292 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,293 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#-2129873378] 03:04:00,294 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#-2129873378] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,294 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 03:04:00,296 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-2131771422 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,296 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-2131771422] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,305 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:00,305 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:00,305 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:00,305 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:04:00,305 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 03:04:00,305 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 03:04:00,305 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 03:04:00,307 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,309 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 03:04:00,316 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-517651869] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,316 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,316 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:00,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:00,327 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:00,337 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 03:04:00,337 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,337 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:04:00,337 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,337 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@18bd9fd6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3e052e89 03:04:00,337 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6092e9a3 03:04:00,337 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1709823840337, None)) 03:04:00,337 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823840336,0,0)) 03:04:00,338 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3e052e89 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5015a6ca 03:04:00,338 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 03:04:00,338 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@c664338 03:04:00,338 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:00,338 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,340 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:00,342 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:00,342 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,342 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,343 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 03:04:00,343 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 03:04:00,347 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#1195903969] 03:04:00,347 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#1179056899] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,347 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#1195903969] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,348 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#2145279448 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,348 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#2145279448] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:00,350 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:00,350 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:00,350 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 03:04:00,350 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 03:04:00,350 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:00,350 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:04:00,350 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 03:04:00,351 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,351 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,351 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 03:04:00,388 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-1334993715] 03:04:00,389 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 03:04:00,389 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416] 03:04:00,566 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:00,566 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 03:04:00,566 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 03:04:00,566 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,567 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 03:04:00,567 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:00,567 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 03:04:00,567 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:00,567 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:04:00,567 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 03:04:00,567 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 03:04:00,806 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:00,806 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:00,806 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 03:04:00,953 PM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:01,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:01,126 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,126 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 03:04:01,207 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:01,207 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:01,208 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:01,208 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,208 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:01,208 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,208 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,208 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,239 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:01,240 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,240 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,240 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:01,240 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,240 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,246 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:01,366 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:01,366 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,366 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 03:04:01,366 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,367 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:01,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 03:04:01,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:01,368 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:01,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=null 03:04:01,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:01,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,368 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 03:04:01,374 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 03:04:01,374 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 03:04:01,375 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:04:01,375 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,375 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 03:04:01,375 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=13] 03:04:01,375 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:01,375 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:01,375 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 03:04:01,375 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 03:04:01,376 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:01,394 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6a79b071: creating a new snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 03:04:01,394 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 03:04:01,394 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:01,394 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:01,395 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-517651869] 03:04:01,395 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=0} shard 1 03:04:01,395 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:01,395 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 03:04:01,395 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:01,395 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@c1595fe 03:04:01,395 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 03:04:01,395 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} scheduling timeout in 2 seconds 03:04:01,395 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=13] 03:04:01,395 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@530686fa, attempting to connect 03:04:01,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} is not yet known 03:04:01,397 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=10550, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:04:01,398 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:01,398 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@423f5cd1 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@441f9d40 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@441f9d40 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2089/0x00000008016bf9a8@3b6f991 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1995910348 nanoseconds 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e517a5f transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=887801060933} 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6a79b071 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@46b07890 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7efd189b superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4a330ca4, splicing queue 03:04:01,399 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7efd189b queue spliced 0 messages 03:04:01,400 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 883.8 μs 03:04:01,400 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: allocating new open snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 03:04:01,401 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-340: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 03:04:01,401 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-340: Message size: 239 bytes, total slices to send: 1 03:04:01,401 PM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-340: Message does not need to be sliced - sending original message 03:04:01,402 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 03:04:01,402 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=887801060933, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 03:04:01,402 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0} 03:04:01,402 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:04:01,403 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-343: Closing 03:04:01,403 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 03:04:01,403 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,404 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:01,404 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:01,409 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:01,409 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:01,409 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:01,409 PM [cluster-test-akka.actor.default-dispatcher-16] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:01,412 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:01,418 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 03:04:01,519 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,519 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6a79b071: creating a new transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 03:04:01,519 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 03:04:01,519 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:01,519 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6a79b071 readied transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 03:04:01,519 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@303c0099 03:04:01,519 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:01,519 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e517a5f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], modifications=1, protocol=SIMPLE}, enqueuedTicks=887925483868} 03:04:01,519 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-341: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], modifications=1, protocol=SIMPLE}} 03:04:01,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,520 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-341: Message size: 402 bytes, total slices to send: 1 03:04:01,520 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-341: Message does not need to be sliced - sending original message 03:04:01,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:04:01,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 03:04:01,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 03:04:01,520 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:04:01,693 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 03:04:01,693 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 03:04:01,693 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 03:04:01,693 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 03:04:01,693 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 03:04:01,693 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,693 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 03:04:01,693 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,694 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:04:01,694 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 03:04:01,694 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:01,694 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 03:04:01,694 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 03:04:01,694 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 03:04:01,694 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:01,694 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e517a5f superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@3f45043f, splicing queue 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1825194347 nanoseconds 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7e517a5f queue spliced 1 messages 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: refreshing backend for shard 1 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:01,694 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-517651869] 03:04:01,695 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:01,695 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 03:04:01,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:01,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=6, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,893 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 6 03:04:01,893 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:01,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 03:04:01,915 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 03:04:01,916 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:01,917 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:01,917 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:01,918 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:02,53 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:02,53 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=7, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,53 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 7 03:04:02,53 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,243 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:02,243 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=8, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,243 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 8 03:04:02,243 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,393 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:02,393 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=9, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,393 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 9 03:04:02,393 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,416 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7771689472981444931] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:02,423 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [7771689472981444931] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:02,572 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:02,572 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=10, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,573 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 10 03:04:02,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,713 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:02,713 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=11, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,713 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 11 03:04:02,713 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,923 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:02,923 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=12, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,923 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 12 03:04:02,923 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:02,933 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:02,933 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:02,933 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:02,933 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-517651869] 03:04:02,934 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:02,934 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 03:04:03,113 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:03,113 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=13, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:03,113 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 13 03:04:03,114 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:03,152 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 03:04:03,152 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 03:04:03,153 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:03,153 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:03,153 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:03,153 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:03,312 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:03,312 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=14, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:03,312 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 14 03:04:03,313 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:03,413 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 03:04:03,413 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} is empty 03:04:03,413 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 03:04:03,413 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:03,413 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 03:04:03,413 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:03,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 03:04:03,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=15, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:03,443 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 15 03:04:03,443 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:03,543 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:03,543 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2077337027], modifications=1, protocol=SIMPLE}, enqueuedTicks=887925483868, sessionId=0, txSequence=1} 03:04:03,544 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 03:04:03,544 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:03,547 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} directCommit completed 03:04:03,547 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED}: initiating purge 03:04:03,547 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 03:04:03,547 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2077337027]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@13f04e4b 03:04:03,547 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 03:04:03,547 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999949839 nanoseconds 03:04:03,548 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:04:03,548 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:03,548 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:04:03,548 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 03:04:03,548 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:03,548 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:04:03,551 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 03:04:03,551 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 03:04:03,551 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 03:04:03,551 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 03:04:03,551 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:03,551 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:04,566 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:04,566 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:04,566 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:04,567 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2846220367935070154] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/07/2024 15:04:04.568] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:04,574 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:04,574 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:04,576 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:04.577] [cluster-test-akka.actor.internal-dispatcher-5] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:04,597 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:04,604 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2288682052573725635] 03:04:04,604 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:04,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:04,606 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:04,613 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:04,613 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:04,621 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:04,621 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:04,621 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:04,624 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:04,632 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-72224889660311857] 03:04:04,632 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:04,632 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:04,632 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:04,640 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:04,641 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:04,652 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:04,676 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-6638669357481726961] 03:04:04,676 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:04,684 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:04,684 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:04,686 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:04,690 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:04,692 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:04,692 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:04,694 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:04,695 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:04,697 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 03:04:04,700 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,700 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 03:04:04,702 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 03:04:04,702 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:04:04,702 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3b77bb61 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@25e9b52d 03:04:04,702 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 03:04:04,702 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,703 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:04,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,703 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,707 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@a8600ab 03:04:04,707 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1709823844702, None)) 03:04:04,707 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823844701,0,0)) 03:04:04,708 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@320debba 03:04:04,708 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@25e9b52d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5b58442e 03:04:04,708 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 03:04:04,708 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:04,708 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,708 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:04,710 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:04,712 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 03:04:04,723 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,723 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,724 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613] 03:04:04,724 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,724 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 03:04:04,725 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 03:04:04,725 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 03:04:04,725 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 03:04:04,725 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 03:04:04,725 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,725 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,726 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1096856839 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,726 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#397281698] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,726 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1096856839] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,726 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1251216653] 03:04:04,726 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#1600652334] 03:04:04,726 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,726 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,726 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240] 03:04:04,727 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#1600652334] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,727 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1251216653] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,730 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-1979054835 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,730 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-1979054835] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:04,730 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:04,730 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:04,730 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:04,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:04,730 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:04,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:04:04,730 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:04:04,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 03:04:04,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 03:04:04,730 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 03:04:04,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 03:04:04,730 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 03:04:04,730 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 03:04:04,730 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 03:04:04,730 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 03:04:04,730 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 03:04:04,730 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 03:04:04,731 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 03:04:04,731 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:04:04,731 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 03:04:04,731 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@47fb8194 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@32a262e9 03:04:04,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,731 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3fd858a1 03:04:04,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,731 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1709823844731, None)) 03:04:04,731 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823844730,0,0)) 03:04:04,732 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@32a262e9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7828c0ec 03:04:04,732 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 03:04:04,736 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:04,736 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,736 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,736 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1002d 03:04:04,736 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:04,736 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,737 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#2137382904] 03:04:04,737 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,738 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 03:04:04,738 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,738 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,738 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#143033222] 03:04:04,738 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 03:04:04,738 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 03:04:04,739 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 03:04:04,739 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#570401976] 03:04:04,739 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-746303733 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,739 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#2137382904] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,739 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1220274475 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,740 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1335677050] 03:04:04,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:04,740 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:04,740 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 03:04:04,740 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 03:04:04,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:04,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:04:04,740 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:04,740 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:04,740 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:04,740 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 03:04:04,740 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 03:04:04,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 03:04:04,740 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 03:04:04,740 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 03:04:04,741 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#143033222] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,741 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-746303733] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,741 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#570401976] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,741 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-1220274475] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,741 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1335677050] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:04,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:04,741 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:04:04,741 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 03:04:04,741 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,742 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 03:04:04,742 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 03:04:04,754 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:04,756 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:04,782 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] 03:04:04,782 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 03:04:04,783 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613] 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:04:04,942 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 03:04:04,942 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 03:04:05,11 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:05,11 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:04:05,11 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 03:04:05,11 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,11 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 03:04:05,12 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:05,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 03:04:05,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:05,12 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 03:04:05,12 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 03:04:05,12 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 03:04:05,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:04:05,181 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,181 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 03:04:05,262 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:05,281 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:04:05,281 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,281 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 03:04:05,332 PM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:05,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:04:05,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,422 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 03:04:05,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:04:05,592 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,592 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 03:04:05,632 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:05,632 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,633 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,640 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,672 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:04:05,672 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,672 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 03:04:05,672 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:05,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 03:04:05,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:05,673 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:05,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 03:04:05,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:05,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,673 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 03:04:05,681 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:05,681 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 03:04:05,682 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 03:04:05,682 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,683 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 03:04:05,683 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:05,683 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:05,683 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 03:04:05,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:04:05,683 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:05,707 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:05,739 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] 03:04:05,739 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 03:04:05,739 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240] 03:04:05,793 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 03:04:05,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 03:04:05,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,861 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 03:04:05,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:05,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 03:04:05,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:05,863 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:05,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 03:04:05,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:05,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,863 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 03:04:05,864 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 03:04:05,864 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 03:04:05,865 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 03:04:05,865 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,865 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=13] 03:04:05,865 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:05,865 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:05,865 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 03:04:05,865 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 03:04:05,865 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:04:05,865 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:05,890 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@56a3ef7 03:04:05,891 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 362721334, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@209f99e4} 03:04:05,891 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@56a3ef7 changed state from IDLE to TX_OPEN 03:04:05,891 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 03:04:05,891 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 03:04:05,891 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 03:04:05,891 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#397281698] 03:04:05,891 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:05,891 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 03:04:05,891 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 03:04:05,891 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:05,891 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 03:04:05,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 03:04:05,892 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:05,892 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 03:04:05,892 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5dfae80f, attempting to connect 03:04:05,892 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 03:04:05,892 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 03:04:05,892 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#397281698] 03:04:05,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:05,892 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 03:04:05,892 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 03:04:05,892 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 03:04:05,892 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 03:04:05,892 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 03:04:05,892 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@7fa64429 03:04:05,892 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@2bef40ac 03:04:05,892 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@56a3ef7 changed state from TX_OPEN to IDLE 03:04:05,892 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=13] 03:04:05,892 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@56a3ef7 readied transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 03:04:05,892 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 03:04:05,893 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 03:04:05,893 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@77e3ac8c 03:04:05,893 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 03:04:05,893 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3dd7a707, attempting to connect 03:04:05,893 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@222854fc 03:04:05,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 03:04:05,894 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 03:04:05,894 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=14101, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:04:05,894 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=10251, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:04:05,895 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:05,895 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:05,895 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 03:04:05,895 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 03:04:05,895 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@164124f6 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a055681 03:04:05,895 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:04:05,895 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:04:05,895 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10d23de2 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892297528801} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4a055681 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2043/0x00000008016a3198@7dc9a029 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10d23de2 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY}, enqueuedTicks=892298467844} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY}} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: Message size: 406 bytes, total slices to send: 1 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: Message does not need to be sliced - sending original message 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@32c3bbf1 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10d23de2 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=892298747762} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE}} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: Message size: 298 bytes, total slices to send: 1 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: Message does not need to be sliced - sending original message 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@56a3ef7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2ddfd7f2 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@48bca455 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1b6d698a, splicing queue 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@48bca455 queue spliced 0 messages 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} in 1.047 ms 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1a37adc1 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@37842cde 03:04:05,896 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4755229d transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892298222898} 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@37842cde 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2043/0x00000008016a3198@2a268710 03:04:05,897 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4755229d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY}, enqueuedTicks=892298534636} 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 03:04:05,897 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 03:04:05,897 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 03:04:05,897 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3782fc1c 03:04:05,897 PM [cluster-test-shard-dispatcher-26] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 ready 03:04:05,897 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 03:04:05,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transitioned member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 03:04:05,898 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 03:04:05,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 03:04:05,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 03:04:05,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Validating transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 03:04:05,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 validated 03:04:05,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:05,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 03:04:05,898 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY}} 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: Message size: 491 bytes, total slices to send: 1 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: Message does not need to be sliced - sending original message 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@3718fdb3 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4755229d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=892298828574} 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE}} 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: Message size: 299 bytes, total slices to send: 1 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-347: Message does not need to be sliced - sending original message 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@56a3ef7 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@25171d5d 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2170ed43 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@793ece62, splicing queue 03:04:05,898 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2170ed43 queue spliced 0 messages 03:04:05,899 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} in 2.056 ms 03:04:05,899 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 03:04:05,899 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY}, enqueuedTicks=892298467844, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 03:04:05,899 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 03:04:05,899 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 03:04:05,899 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=892298747762, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 03:04:05,899 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 03:04:05,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 03:04:05,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 03:04:05,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5393792e 03:04:05,899 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:05,900 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 03:04:05,901 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215], modifications=0, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:05,911 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215], modifications=1, protocol=READY} failed to process}} 03:04:05,911 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY}, enqueuedTicks=892298534636, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215], modifications=1, protocol=READY} failed to process}} 03:04:05,911 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215], modifications=1, protocol=READY} failed to process} 03:04:05,911 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 03:04:05,911 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 03:04:05,911 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-135654215], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=892298828574, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 03:04:05,912 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:05,912 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@76ebdf41 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1127) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.resolveResult(VotingFuture.java:91) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.voteNo(VotingFuture.java:67) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$canCommit$5(AbstractProxyTransaction.java:549) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$17(RemoteProxyTransaction.java:516) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 03:04:05,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,914 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 03:04:05,915 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2106/0x00000008016c7d70@28b3b519 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2106/0x00000008016c7d70@28b3b519 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10d23de2 transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892322152535} 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2106/0x00000008016c7d70@3b820340 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2106/0x00000008016c7d70@3b820340 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4755229d transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892322404231} 03:04:05,916 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 03:04:05,918 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=3, message=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215]}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:05,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborting 03:04:05,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: aborting head of queue member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 in state member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 03:04:05,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1: abort - successfulFromPrevious: [] 03:04:05,920 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborted 03:04:05,919 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 03:04:05,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 03:04:05,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:05,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 03:04:05,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 03:04:05,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 03:04:05,920 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:05,922 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 03:04:05,923 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@76ebdf41 is no longer running 03:04:05,923 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:04:05,923 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 03:04:05,924 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 03:04:05,924 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6df718f1 03:04:05,924 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3313e211 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6df718f1 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 03:04:05,924 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:05,924 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3313e211 03:04:05,924 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 03:04:05,924 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}]]] 03:04:05,924 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:05,924 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:05,924 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:04:05,924 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:05,924 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:05,924 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:04:05,924 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:05,924 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:05,927 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}]]] 03:04:05,927 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 03:04:05,927 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:05,927 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 03:04:05,927 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 03:04:05,927 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 03:04:05,927 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:05,927 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards] 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892322404231, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} abort completed 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@6199105f 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 3 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@35d47808 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4755229d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892333331344} 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 03:04:05,927 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892322152535, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [ERROR] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during Abort. org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} abort completed 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@2bc9939a 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 3 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@2fb6554 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10d23de2 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892333774276} 03:04:05,928 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 03:04:05,928 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards] 03:04:05,928 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:04:05,928 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:05,928 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 03:04:05,928 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:05,928 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4755229d superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@14f9493b, splicing queue 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4755229d queue spliced 2 messages 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: refreshing backend for shard 2 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#397281698] 03:04:05,928 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:05,928 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:05,928 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10d23de2 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@2327c191, splicing queue 03:04:05,929 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:04:05,929 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@10d23de2 queue spliced 2 messages 03:04:05,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 03:04:05,929 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: refreshing backend for shard 1 03:04:05,929 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 03:04:05,929 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 03:04:05,929 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 03:04:05,929 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#397281698] 03:04:05,929 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:05,929 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:05,929 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:04:05,929 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:05,929 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 03:04:05,929 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 03:04:05,929 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2048/0x00000008016a84c0@7d3afcf8 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3c07f6c3}, purged=MutableUnsignedLongSet{size=0}} 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, matchIndex: 0, lastIndex: 0, isVoting: true 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 03:04:05,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} not found in inflight queue, checking pending queue 03:04:05,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:04:05,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892298222898, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 03:04:05,930 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 03:04:05,930 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: not currently active leader, rejecting request Envelope{sessionId=1, txSequence=4, message=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215]}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 03:04:05,931 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 03:04:05,931 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:05,931 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 03:04:05,931 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 03:04:05,931 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 03:04:05,931 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=4, message=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215]}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:05,931 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@d0f0f30}, purged=MutableUnsignedLongSet{size=0}} 03:04:05,931 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 03:04:05,932 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2048/0x00000008016a84c0@28cf705c 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@79ff9415}, purged=MutableUnsignedLongSet{size=0}} 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testChainedTransactionFailureWithMultipleShards, matchIndex: 0, lastIndex: 0, isVoting: true 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leader transfer complete 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=4, message=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215]}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 03:04:05,932 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=0, txSequence=4, message=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-135654215]}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#18084240] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:05,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 03:04:05,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:05,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 03:04:05,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 03:04:05,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77} 03:04:05,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@29713844}, purged=MutableUnsignedLongSet{size=0}} 03:04:05,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 03:04:05,934 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:04:05,934 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} not found in inflight queue, checking pending queue 03:04:05,934 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 03:04:05,934 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-135654215]}, enqueuedTicks=892297528801, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 03:04:05,934 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 03:04:06,154 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 214, lastApplied : 0, commitIndex : 0 03:04:06,342 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 03:04:06,342 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1916397613]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:06,343 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:06,343 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-350: Closing 03:04:06,343 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 03:04:06,344 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:06,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-351: Closing 03:04:06,344 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 03:04:06,344 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 03:04:06,344 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 03:04:06,344 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 03:04:06,345 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 03:04:06,345 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 03:04:06,345 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 03:04:06,345 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:06,345 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:06,345 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#147033423] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:06,354 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:06,354 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:06,361 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:06,361 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:06,361 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:06,361 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:06,373 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 03:04:06,374 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 03:04:06,374 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 03:04:06,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 03:04:06,374 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 03:04:06,374 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 03:04:06,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 03:04:06,374 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 03:04:06,375 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:06,375 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:07,367 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2288682052573725635] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:07,387 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-72224889660311857] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:07,393 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:07,394 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:07,395 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:07.396] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:07,404 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:07,404 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:04:07.405] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:07,432 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:07,439 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4880850709429023748] 03:04:07,439 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:07,440 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:07,440 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:07,443 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:07,443 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:07,454 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:07,454 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:07,454 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:07,458 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:07,494 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4659392414315769889] 03:04:07,494 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:07,497 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:07,497 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:07,505 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:07,505 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:07,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:07,548 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3837415151761446176] 03:04:07,549 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:07,551 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:07,551 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:07,562 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:07,563 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:07,565 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:07,567 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:07,576 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:07,577 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 03:04:07,579 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@423f5cd1 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:04:07,580 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:07,580 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2077337027], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@729fc7f3 03:04:07,580 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} has a successor, not scheduling timer 03:04:07,580 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 1 03:04:07,580 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#2077337027], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2107/0x00000008016ccb68@2554a7ff 03:04:07,580 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#261180416], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 03:04:07,580 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0} 03:04:07,581 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:07,582 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1279849357] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,583 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 03:04:07,587 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 03:04:07,587 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 03:04:07,587 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:07,587 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,587 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@219a87b7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e19ee2c 03:04:07,587 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@9791358 03:04:07,588 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709823847587, None)) 03:04:07,588 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,588 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:07,588 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,588 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,588 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823847586,0,0)) 03:04:07,588 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@66b1a4a8 03:04:07,589 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e19ee2c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6fb3d321 03:04:07,589 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 03:04:07,590 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:07,590 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:07,590 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,591 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] 03:04:07,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,591 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670] 03:04:07,591 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 03:04:07,592 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 03:04:07,592 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 03:04:07,592 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:07,592 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 03:04:07,595 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 03:04:07,603 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-950683551 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,603 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-942125196] 03:04:07,604 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1204549248 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,604 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-926385929] 03:04:07,604 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,613 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,613 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-950683551] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,613 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-942125196] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,613 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1204549248] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,613 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-926385929] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,616 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 03:04:07,616 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:07,616 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@748ffd5f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5ba51d36 03:04:07,616 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:07,616 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:07,617 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 03:04:07,617 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 03:04:07,617 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:07,617 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:04:07,617 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:07,617 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:07,617 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 03:04:07,617 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 03:04:07,617 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 03:04:07,617 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:04:07,617 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:07,617 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:04:07,617 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 03:04:07,617 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 03:04:07,617 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:04:07,618 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 03:04:07,622 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 03:04:07,622 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@31572044 03:04:07,622 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709823847617, None)) 03:04:07,623 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1706989146] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,623 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823847616,0,0)) 03:04:07,623 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5ba51d36 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4d668359 03:04:07,624 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 03:04:07,628 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 03:04:07,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:07,628 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@789ccb77 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-2-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983] 03:04:07,629 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 03:04:07,629 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 03:04:07,630 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 03:04:07,630 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 03:04:07,630 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 03:04:07,631 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,631 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1401249875] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,631 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#1404316822 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:07,631 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-1375232990] 03:04:07,631 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#1404316822] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,631 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-857977382 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 03:04:07,631 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-1985298578] 03:04:07,632 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,632 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:07,632 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-2-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1401249875] 03:04:07,636 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-1375232990] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,636 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-857977382] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,636 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-1985298578] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:07,639 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:07,639 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:07,639 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:07,639 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:04:07,639 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,639 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 03:04:07,639 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 03:04:07,639 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 03:04:07,640 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:04:07,645 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:07,651 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:07,651 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 03:04:07,652 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:07,652 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:07,652 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:07,652 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:04:07,652 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,652 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 03:04:07,652 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 03:04:07,652 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 03:04:07,653 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:04:07,653 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 03:04:07,674 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1279849357] 03:04:07,674 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 03:04:07,675 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] 03:04:07,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:07,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 03:04:07,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 03:04:07,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:07,850 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 03:04:07,851 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:07,851 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:04:07,851 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:07,851 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:04:07,851 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:04:07,851 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:04:07,860 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:04:07,860 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 03:04:08,100 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:04:08,100 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,100 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 03:04:08,140 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:04:08,140 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,140 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 03:04:08,340 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:04:08,340 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,340 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 03:04:08,430 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:04:08,430 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,430 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 03:04:08,461 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:08,461 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,462 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,468 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,570 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:04:08,570 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,570 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 03:04:08,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,572 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:08,573 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 03:04:08,573 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:08,573 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:08,573 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 03:04:08,573 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,573 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:08,573 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,573 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:08,574 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:08,574 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:08,574 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 03:04:08,574 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,574 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,574 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:08,575 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:04:08,575 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 03:04:08,575 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:04:08,575 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,575 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:08,575 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:08,575 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 03:04:08,575 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,576 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 03:04:08,576 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,576 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,576 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:08,576 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:08,576 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 03:04:08,576 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:04:08,576 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,580 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:08,680 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 03:04:08,680 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,680 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 03:04:08,680 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,681 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:08,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 03:04:08,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:08,682 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:08,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 03:04:08,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:08,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,682 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,682 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 03:04:08,685 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 03:04:08,685 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:04:08,686 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,686 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:08,686 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,686 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,686 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:08,686 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:08,687 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:04:08,687 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:04:08,687 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:04:08,687 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,691 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,693 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 03:04:08,780 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1279849357] 03:04:08,780 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 03:04:08,780 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670] 03:04:08,781 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:08,783 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:08,786 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 03:04:08,789 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 03:04:08,789 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1777086005] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,792 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 03:04:08,792 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 03:04:08,792 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:08,792 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,792 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@41833bc5 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@148ee72 03:04:08,792 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@70081e64 03:04:08,792 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,792 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:08,792 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,793 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,793 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:08,793 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,793 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,793 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:08,793 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,793 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,793 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1709823848793, None)) 03:04:08,793 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823848792,0,0)) 03:04:08,793 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6b338ed6 03:04:08,793 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@148ee72 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1a8f1743 03:04:08,793 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,794 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,794 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 03:04:08,794 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:08,794 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735] 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,794 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1706989146] 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735] 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735] 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,794 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 03:04:08,794 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:04:08,794 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1739878593] 03:04:08,794 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,794 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,794 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1739878593] 03:04:08,794 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1706989146] 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,794 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1739878593] 03:04:08,794 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 2 03:04:08,794 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:04:08,794 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:04:08,794 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454 readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:04:08,794 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:04:08,795 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:04:08,795 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:08,795 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:04:08,795 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454 readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 03:04:08,795 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 03:04:08,795 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,795 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 03:04:08,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:08,795 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1279849357] 03:04:08,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 03:04:08,795 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1748105714 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:08,795 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1748105714] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,795 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-1817646875] 03:04:08,795 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 03:04:08,796 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-1817646875] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,796 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 03:04:08,796 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,796 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,796 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 03:04:08,796 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 03:04:08,796 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 03:04:08,796 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#-552917340 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 03:04:08,796 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-1545444821] 03:04:08,797 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 03:04:08,797 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 03:04:08,797 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=13] 03:04:08,797 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@57f8d669, attempting to connect 03:04:08,797 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] 03:04:08,797 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 03:04:08,797 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@406be12e, attempting to connect 03:04:08,798 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#-1739878593] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,798 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:08,799 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:08,799 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:08,799 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:04:08,799 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 03:04:08,799 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,799 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:08,799 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 03:04:08,799 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 03:04:08,799 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:04:08,799 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:08,799 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:08,800 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:08,800 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:04:08,800 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 03:04:08,800 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,800 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,800 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=2, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@42ed51fb 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 03:04:08,800 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:08,800 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@7d2fa0e6 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11f4b660 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@11f4b660 03:04:08,800 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@72d99216 skipping transactions [1] 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,800 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} scheduling timeout in 30 seconds 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:08,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:08,800 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 03:04:08,800 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 03:04:08,800 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 03:04:08,800 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 03:04:08,801 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 03:04:08,802 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 03:04:08,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 03:04:08,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:08,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,802 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:08,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:08,802 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 03:04:08,803 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:04:08,803 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:08,803 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:08,803 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:04:08,803 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 03:04:08,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:04:08,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-testLeadershipTransferOnShutdown, matchIndex: -1, lastIndex: -1, isVoting: true 03:04:08,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 03:04:08,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:08,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:08,805 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-359: Closing 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:04:08,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:08,805 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 03:04:08,806 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:04:08,806 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 03:04:08,806 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:08,806 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 03:04:08,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:08,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-3-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,807 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,807 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:08,807 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:08,807 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:04:08,807 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-3-shard-cars-testLeadershipTransferOnShutdown in 7.423 ms 03:04:08,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 03:04:08,807 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:04:08,808 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:08,808 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:04:08,808 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:04:08,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:08,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 03:04:08,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:08,808 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 4 - updating term 03:04:08,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 03:04:08,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:08,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cannot append entries because sender's term 4 is less than 5 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-3-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cannot append entries because sender's term 4 is less than 5 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cannot append entries because sender's term 4 is less than 5 03:04:08,809 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 03:04:08,809 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:08,809 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:04:08,809 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:04:08,809 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:04:08,809 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,809 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.access.commands.ConnectClientRequest] from Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown$a] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,810 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntriesReply] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,810 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 03:04:08,810 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntriesReply] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#940903543] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:08,810 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=9190, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:08,810 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:08,810 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:04:08,810 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:08,810 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@72d99216 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b2a9649 03:04:08,810 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b2a9649 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@345bd92d 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@aa56601 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=895206165521} 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=THREE_PHASE}} 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: Message size: 390 bytes, total slices to send: 1 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: Message does not need to be sliced - sending original message 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6b8cfd87 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3d71ab8b superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@484757bf, splicing queue 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3d71ab8b queue spliced 0 messages 03:04:08,811 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 843.3 μs 03:04:08,812 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 03:04:08,812 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 03:04:08,812 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 03:04:08,812 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 03:04:08,812 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:08,812 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:08,812 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:08,812 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:04:08,812 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: allocating new open transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 ready 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: transitioned member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 to ready 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating canCommit 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 validated 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:08,822 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed canCommit 03:04:08,827 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 03:04:08,827 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=895206165521, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 03:04:08,827 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 03:04:08,922 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 03:04:08,932 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 03:04:09,481 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:04:13,814 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)]] after [5000 ms]. Message of type [org.opendaylight.controller.cluster.access.commands.ConnectClientRequest]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:13,815 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)]] after [5000 ms]. Message of type [org.opendaylight.controller.cluster.access.commands.ConnectClientRequest]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:13,815 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)]] after [5000 ms]. Message of type [org.opendaylight.controller.cluster.access.commands.ConnectClientRequest]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:13,815 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)]] after [5000 ms]. Message of type [org.opendaylight.controller.cluster.access.commands.ConnectClientRequest]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:14,835 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 03:04:14,835 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:14,835 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:14,835 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1706989146] 03:04:14,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:14,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 03:04:14,835 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=13] 03:04:14,835 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6bcce9b6, attempting to connect 03:04:14,838 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 03:04:14,838 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=13710, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:14,839 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b27e295 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6c05f734 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} previous state SEALED 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6c05f734 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=2, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 03:04:14,839 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=2, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@785a0e82 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=2, protocol=THREE_PHASE}, enqueuedTicks=895205902104} 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=2, protocol=THREE_PHASE}} 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: Message size: 403 bytes, total slices to send: 1 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: Message does not need to be sliced - sending original message 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6c05f734 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 03:04:14,840 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 03:04:14,841 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:14,841 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:14,841 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 03:04:14,841 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} 03:04:14,841 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1a28971f 03:04:14,841 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@11bc322d superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2abceed4, splicing queue 03:04:14,841 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@11bc322d queue spliced 0 messages 03:04:14,841 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.892 ms 03:04:14,842 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-3-shard-cars-testLeadershipTransferOnShutdown: allocating new open transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 ready 03:04:14,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: transitioned member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 to ready 03:04:14,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating canCommit 03:04:14,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 validated 03:04:14,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:14,843 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed canCommit 03:04:14,844 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=2, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 03:04:14,844 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=2, protocol=THREE_PHASE}, enqueuedTicks=895205902104, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 03:04:14,844 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 03:04:14,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:14,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@2975cd8e 03:04:14,844 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 03:04:14,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 03:04:14,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@1bf23d2 03:04:14,844 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:14,844 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901250369074} 03:04:14,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 03:04:14,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@1178b93 03:04:14,844 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 03:04:14,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 1 03:04:14,844 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@3fe0e085 03:04:14,845 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:14,845 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@aa56601 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901250664791} 03:04:14,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating preCommit 03:04:14,845 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,846 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 candidate ready 03:04:14,846 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - candidate rootPath: / 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating preCommit 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,846 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - messages: [] 03:04:14,846 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 03:04:14,846 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 prepared 03:04:14,846 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:14,846 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed preCommit 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 candidate ready 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - messages: [] 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 prepared 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:14,846 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed preCommit 03:04:14,846 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 03:04:14,846 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901250664791, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 03:04:14,846 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:14,846 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 03:04:14,846 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901250369074, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 03:04:14,846 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:14,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:04:14,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@387e534a 03:04:14,847 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 03:04:14,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 2 03:04:14,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@387e534a 03:04:14,847 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:14,847 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901252843876} 03:04:14,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 03:04:14,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@40a74624 03:04:14,847 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 03:04:14,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 2 03:04:14,847 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@40a74624 03:04:14,847 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:14,847 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@aa56601 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901253125083} 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating commit 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating commit 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 03:04:14,848 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 submitted to persistence 03:04:14,848 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 03:04:14,849 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 submitted to persistence 03:04:14,849 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 03:04:14,849 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:14,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 03:04:14,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:04:14,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:14,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 03:04:14,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:14,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 03:04:14,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:14,850 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:14,850 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 03:04:14,850 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:04:14,850 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:14,850 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 03:04:14,850 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:14,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 03:04:14,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:04:14,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:14,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=170} 03:04:14,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 03:04:14,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 03:04:14,850 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 03:04:14,850 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:14,850 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:14,851 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 03:04:14,851 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 03:04:14,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:14,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 03:04:14,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:04:14,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6e798f67}, purged=MutableUnsignedLongSet{size=0}} 03:04:14,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@33831f23}, purged=MutableUnsignedLongSet{size=0}} 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: commit - successfulFromPrevious: [] 03:04:14,854 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 03:04:14,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:04:14,854 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 03:04:14,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:04:14,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-323226078] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 03:04:14,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1294632175] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 03:04:14,855 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-323226078] 03:04:14,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:14,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:14,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 03:04:14,855 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:04:14,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:14,855 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:04:14,855 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:04:14,855 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1294632175] 03:04:14,855 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:04:14,855 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 163.5 μs 03:04:14,855 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 78.48 μs 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901253125083, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@72d99216 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@52a8b0d1 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED}: initiating purge 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} allocated sequence 3 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@c6655f4 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:14,856 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@aa56601 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901262132508} 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@16617409}, purged=MutableUnsignedLongSet{size=0}} 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1503228406] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 03:04:14,856 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1503228406] 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 54.90 μs 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1503228406] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:14,857 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 03:04:14,857 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:14,857 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 03:04:14,857 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:04:14,857 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901252843876, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 03:04:14,857 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b27e295 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 03:04:14,857 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 03:04:14,857 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 03:04:14,857 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@18f2801d 03:04:14,857 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED}: initiating purge 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} allocated sequence 3 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@3d74eb47 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:14,857 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901263382649} 03:04:14,857 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@7d5116b0 03:04:14,857 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=SIMPLE}, enqueuedTicks=901263551643} 03:04:14,857 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=SIMPLE}} 03:04:14,858 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: Message size: 448 bytes, total slices to send: 1 03:04:14,858 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-355: Message does not need to be sliced - sending original message 03:04:14,858 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,858 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,858 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,858 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 03:04:14,858 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,858 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 03:04:14,858 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 03:04:14,858 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 03:04:14,858 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:14,859 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-3-shard-cars-testLeadershipTransferOnShutdown: allocating new open transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,859 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@456d1912}, purged=MutableUnsignedLongSet{size=0}} 03:04:14,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@37085cc9}, purged=MutableUnsignedLongSet{size=0}} 03:04:14,859 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 03:04:14,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 03:04:14,859 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 ready 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: transitioned member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 to ready 03:04:14,859 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1314861703] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 03:04:14,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1289302144] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct canCommit 03:04:14,859 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1289302144] 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 validated 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:14,859 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:04:14,859 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:04:14,859 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct preCommit 03:04:14,859 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,859 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1314861703] 03:04:14,859 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 70.14 μs 03:04:14,859 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1289302144] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:14,859 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:04:14,859 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 56.16 μs 03:04:14,860 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 candidate ready 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - candidate rootPath: / 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - messages: [] 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 prepared 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct commit 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}], SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:14,860 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 submitted to persistence 03:04:14,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 03:04:14,861 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 03:04:14,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 03:04:14,861 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}], SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=229} 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 03:04:14,862 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 03:04:14,863 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 03:04:14,863 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 03:04:14,863 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 03:04:14,863 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:04:14,863 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 03:04:14,863 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@146ac49a 03:04:14,863 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 03:04:14,863 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:04:14,863 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:14,863 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 03:04:14,863 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 03:04:14,863 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-3-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,863 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 03:04:14,863 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: commit - successfulFromPrevious: [] 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 03:04:14,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:04:14,864 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 82.95 μs 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:14,864 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 03:04:14,864 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@b3c9341 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 03:04:14,866 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901263382649, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE}: purge completed 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6c05f734 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b27e295 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=1, protocol=SIMPLE}, enqueuedTicks=901263551643, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} directCommit completed 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED}: initiating purge 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} allocated sequence 1 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@42e4b4ed 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:14,866 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901272492886} 03:04:14,866 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454: creating a new snapshot member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 03:04:14,867 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 03:04:14,867 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} allocated sequence 0 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 03:04:14,867 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@31a9cf0c 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:04:14,867 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=901272830054} 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 03:04:14,867 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 03:04:14,867 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,868 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 03:04:14,868 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 03:04:14,868 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 88.59 μs 03:04:14,868 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} 03:04:14,868 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901262132508, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3}} 03:04:14,868 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE}: purge completed 03:04:14,868 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b2a9649 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} 03:04:14,868 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 03:04:14,868 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@72d99216 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 03:04:14,869 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,869 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 03:04:14,869 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 03:04:14,869 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:14,869 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-3-shard-cars-testLeadershipTransferOnShutdown: allocating new open snapshot member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 03:04:14,869 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-3-shard-cars-testLeadershipTransferOnShutdown_slicer-id-361: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, message: Envelope{sessionId=1, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0}} 03:04:14,869 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 03:04:14,869 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-3-shard-cars-testLeadershipTransferOnShutdown_slicer-id-361: Message size: 365 bytes, total slices to send: 1 03:04:14,869 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-3-shard-cars-testLeadershipTransferOnShutdown_slicer-id-361: Message does not need to be sliced - sending original message 03:04:14,870 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=1, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0}} 03:04:14,870 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=901272830054, sessionId=1, txSequence=6} with Envelope{sessionId=1, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0}} 03:04:14,870 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0} 03:04:14,870 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@43b9ccc8: creating a new snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:04:14,870 PM [main] [DEBUG] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 03:04:14,870 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:14,870 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:14,870 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1777086005] 03:04:14,870 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:14,870 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 03:04:14,870 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@372957ff] 03:04:14,870 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 03:04:14,870 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:14,870 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-128796272], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@40853e2d 03:04:14,870 PM [main] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:14,870 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@372957ff] 03:04:14,871 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@652bccee, attempting to connect 03:04:14,871 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 03:04:14,871 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=8960, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:14,871 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b0acb84 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@45670808 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@45670808 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-128796272], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@43b9ccc8 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@74f8cfca 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@118cc44c superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@c477b12, splicing queue 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@118cc44c queue spliced 0 messages 03:04:14,871 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 359.7 μs 03:04:14,872 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0} 03:04:14,872 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@717509ae 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@717509ae 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 03:04:14,872 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:04:14,873 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:14,873 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:14,873 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@65254ffa superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@2027cd3a, splicing queue 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@65254ffa queue spliced 0 messages 03:04:14,873 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 1 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:14,873 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1777086005] 03:04:14,873 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:14,873 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Leader, leaderId=null, activeMember=true] 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 4 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 03:04:14,878 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown] 03:04:14,879 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:04:14,879 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 3, nextIndex: 4 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 3 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@adadbc5 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@72f972c9, splicing queue 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-3-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@438242a8 queue spliced 1 messages 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 3, lastIndex: 3, isVoting: true 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 1 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 03:04:14,879 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1706989146] 03:04:14,879 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 03:04:14,879 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:14,879 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:14,880 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} not found in inflight queue, checking pending queue 03:04:14,880 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 03:04:14,880 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]}, enqueuedTicks=901272492886, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 03:04:14,880 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE}: purge completed 03:04:14,880 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6c05f734 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} 03:04:14,880 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 3 03:04:14,880 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 03:04:14,880 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 3 03:04:14,880 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 03:04:14,880 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 03:04:14,880 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 6 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=3, lastLogTerm=5] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=3, lastLogTerm=5] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 6 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:14,881 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 03:04:14,881 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 03:04:14,882 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=3, lastLogTerm=5] with higher term appears reachable, updating term. 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=3, lastLogTerm=5]" message is greater than leader's term 5 - switching to Follower 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=null 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=3, lastLogTerm=5] - currentTerm: 6, votedFor: null, lastIndex: 3, lastTerm: 5 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 6 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-363: Closing 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:14,886 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 03:04:14,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:14,887 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 03:04:14,887 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:04:14,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 03:04:14,887 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 03:04:14,887 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 03:04:14,887 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 03:04:14,887 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 03:04:14,887 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 03:04:14,887 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:14,887 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:14,887 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 03:04:14,887 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:04:14,887 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 03:04:14,888 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:04:14,888 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:14,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 03:04:14,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:14,888 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 03:04:14,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 03:04:14,888 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@735c6c84] 03:04:14,888 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:04:14,888 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:14,888 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@735c6c84] 03:04:14,888 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@18cc6611, attempting to connect 03:04:14,888 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:14,888 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:14,888 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:14,888 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6c05f734 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@428c2683 03:04:14,888 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 03:04:14,888 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@428c2683 03:04:14,889 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 03:04:14,889 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} 03:04:14,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], snapshotOnly=true} 03:04:14,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@488667bc to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} 03:04:14,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 replayed 1 successful requests 03:04:14,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:14,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=SUCCESSOR} 03:04:14,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@61e3201e 03:04:14,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@373203ac superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@719fe349, splicing queue 03:04:14,890 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@373203ac queue spliced 0 messages 03:04:14,891 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.120 ms 03:04:14,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 03:04:14,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:14,891 PM [cluster-test-shard-dispatcher-21] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 18.93 ms 03:04:14,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 03:04:14,891 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 03:04:14,891 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 03:04:14,891 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 03:04:14,891 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:04:14,892 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 03:04:14,892 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 03:04:14,892 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:04:14,892 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:14,893 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 03:04:14,893 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:04:14,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 03:04:14,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 03:04:14,893 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 03:04:14,893 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:04:14,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:14,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 3, nextIndex: 4 03:04:14,893 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37915477 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37915477 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 03:04:14,893 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 03:04:14,893 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62} 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 03:04:14,895 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 03:04:14,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 03:04:14,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 03:04:14,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown] 03:04:14,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 03:04:14,896 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=13] 03:04:14,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testLeadershipTransferOnShutdown, matchIndex: 1, lastIndex: 1, isVoting: true 03:04:14,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 03:04:14,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,896 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:14,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,896 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@aa56601 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@a5fc141, splicing queue 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@aa56601 queue spliced 0 messages 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 2 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:04:14,896 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1706989146] 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testLeadershipTransferOnShutdown 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] to peer member-3-shard-people-testLeadershipTransferOnShutdown 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:04:14,897 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 03:04:14,898 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 03:04:14,898 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:04:14,898 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 03:04:14,898 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 03:04:14,898 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-358: Closing 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 03:04:14,898 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Leader to Follower 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 03:04:14,898 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Leader to Follower 03:04:14,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 03:04:14,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 03:04:14,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 03:04:14,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,899 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:04:14,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 03:04:14,899 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=13] 03:04:14,899 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 03:04:14,899 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized - notifying 1 OnShardInitialized callbacks 03:04:14,899 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:14,899 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 03:04:14,899 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 03:04:14,900 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 03:04:14,900 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is ready - notifying 1 OnShardInitialized callbacks 03:04:14,900 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1401249875, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2f3eb3a1] 03:04:14,900 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 03:04:14,900 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown: Closing 03:04:14,900 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1401249875, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2f3eb3a1] 03:04:14,900 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 03:04:14,900 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-people-testLeadershipTransferOnShutdown in 6.954 ms 03:04:14,900 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 03:04:14,900 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-people-testLeadershipTransferOnShutdown status sync done false 03:04:14,900 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@567583c7, attempting to connect 03:04:14,901 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:14,901 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 03:04:14,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1401249875], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present} 03:04:14,901 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1401249875], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b2a9649 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e341a0e 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@425fb454 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1401249875], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@699084f9 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@50c6cc38 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@597262af, splicing queue 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@50c6cc38 queue spliced 0 messages 03:04:14,901 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 03:04:14,901 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1034910670], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#-1401249875], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 380.2 μs 03:04:14,902 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 03:04:14,902 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:14,902 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:04:14,902 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 03:04:14,902 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 03:04:14,903 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:04:14,903 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:04:14,904 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:14,904 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:14,909 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:14,909 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:14,909 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:14,909 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:14,910 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:14,916 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 03:04:14,917 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 03:04:14,917 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 03:04:14,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-364: Closing 03:04:14,917 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-2-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-365: Closing 03:04:14,917 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:04:14,917 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 03:04:14,917 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 03:04:14,917 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:14,918 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:15,920 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4880850709429023748] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:15,931 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4659392414315769889] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:15,936 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:15,936 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:15,937 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:15.939] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:15,946 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:15,946 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:04:15.948] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:15,971 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:15,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-374975708589141286] 03:04:15,980 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:15,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:15,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:15,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:15,983 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:15,994 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:15,994 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:15,994 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:15,998 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:16,3 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7678148366176179230] 03:04:16,3 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:16,12 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:16,12 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:16,13 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:16,13 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:16,28 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:04:16,35 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4795856837664905782] 03:04:16,35 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:16,35 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:16,35 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:16,42 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:16,44 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:16,45 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:16,45 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:16,47 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 03:04:16,51 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,54 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 03:04:16,55 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 03:04:16,55 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:04:16,55 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5d26744a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5919fb36 03:04:16,55 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1c1b7e71 03:04:16,55 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1709823856055, None)) 03:04:16,55 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 03:04:16,55 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,55 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,55 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:16,56 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823856054,0,0)) 03:04:16,56 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,56 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,56 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2154375 03:04:16,56 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5919fb36 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2a70f4a0 03:04:16,56 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 03:04:16,57 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:16,58 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:16,60 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,61 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] 03:04:16,62 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 03:04:16,62 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 03:04:16,62 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 03:04:16,62 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-59689155 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,63 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1978288942] 03:04:16,63 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 03:04:16,65 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:16,65 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,65 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,65 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-59689155] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,65 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1978288942] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,66 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,66 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 03:04:16,66 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:16,66 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:16,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:16,67 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-991534756 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,67 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:16,67 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-991534756] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:16,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:16,67 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 03:04:16,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 03:04:16,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 03:04:16,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 03:04:16,67 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 03:04:16,69 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:16,69 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:16,69 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 03:04:16,69 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 03:04:16,69 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 03:04:16,73 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 03:04:16,73 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,74 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,74 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:16,74 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 03:04:16,74 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:04:16,74 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4b6df7a6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e6f60af 03:04:16,74 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@63f12e98 03:04:16,74 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:16,74 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,75 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,75 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-287309798] 03:04:16,76 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 03:04:16,76 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 03:04:16,76 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-287309798] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1709823856075, None)) 03:04:16,76 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823856074,0,0)) 03:04:16,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7cb00fa3 03:04:16,77 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e6f60af to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3fdff789 03:04:16,77 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 03:04:16,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:16,77 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,78 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:16,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,78 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998] 03:04:16,78 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 03:04:16,78 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 03:04:16,78 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 03:04:16,79 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-618530461 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,79 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-663440907] 03:04:16,79 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1237369995 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,79 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#869544669] 03:04:16,79 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 03:04:16,79 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:16,79 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:16,79 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 03:04:16,79 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 03:04:16,79 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:16,79 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:16,79 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 03:04:16,79 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,79 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 03:04:16,80 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:16,80 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:16,80 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 03:04:16,80 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 03:04:16,80 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:16,80 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:16,80 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,80 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 03:04:16,80 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 03:04:16,80 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 03:04:16,84 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:16,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-618530461] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-663440907] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1237369995] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,84 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#869544669] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:16,84 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 03:04:16,96 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:16,127 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:16,127 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:16,128 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:16,344 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 03:04:16,344 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:16,374 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:16,374 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 03:04:16,375 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 03:04:16,664 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:04:16,664 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,664 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 03:04:16,664 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:04:16,664 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,664 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 03:04:16,666 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:16,676 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:16,913 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:04:16,913 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,914 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 03:04:16,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:04:16,944 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,944 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 03:04:16,995 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:16,995 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:16,996 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,3 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:17,3 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,4 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,4 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,4 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,4 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:17,4 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,4 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,4 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,4 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,56 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:17,204 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:04:17,204 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,204 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 03:04:17,204 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,206 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:17,206 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 03:04:17,206 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:17,206 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:17,206 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 03:04:17,206 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:17,206 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,206 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 03:04:17,208 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:17,208 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 03:04:17,210 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:17,210 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,210 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:04:17,211 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:17,211 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:17,211 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 03:04:17,211 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 03:04:17,211 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:17,214 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 03:04:17,214 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,214 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 03:04:17,214 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,215 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:17,215 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 03:04:17,215 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:17,215 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:17,215 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 03:04:17,215 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:17,215 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,215 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 03:04:17,216 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 03:04:17,216 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:04:17,217 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:17,217 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,217 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 03:04:17,218 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:04:17,218 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:17,218 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:17,218 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:04:17,218 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 03:04:17,218 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:17,285 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:17,285 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 03:04:17,285 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] 03:04:17,285 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:17,285 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,285 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:17,285 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:17,285 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,285 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,285 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,285 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,286 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:04:17,286 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 03:04:17,286 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:17,286 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:17,286 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,286 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 03:04:17,286 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:04:17,286 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:17,286 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 03:04:17,286 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:04:17,286 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:04:17,286 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,286 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 03:04:17,286 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:04:17,286 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:04:17,286 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:04:17,286 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@64a968b5 03:04:17,286 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:17,286 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@408fd30e 03:04:17,286 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 03:04:17,286 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:17,286 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:17,287 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:17,287 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:17,287 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:17,287 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:17,287 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@74484193, attempting to connect 03:04:17,287 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@335c48bb, attempting to connect 03:04:17,288 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 03:04:17,288 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=10080, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:04:17,288 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 03:04:17,288 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=6010, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:04:17,289 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:17,289 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:17,289 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:04:17,289 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,289 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2a5d5fa1 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 03:04:17,289 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 03:04:17,289 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@52de8398 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43f9b357 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903692513211} 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}} 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message size: 398 bytes, total slices to send: 1 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message does not need to be sliced - sending original message 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@c062d46 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@732e5f65 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7797235c, splicing queue 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@732e5f65 queue spliced 0 messages 03:04:17,290 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 992.2 μs 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@378e6b23 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@365adfc0 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@365adfc0 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@78198794 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903692442020} 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}} 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message size: 395 bytes, total slices to send: 1 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message does not need to be sliced - sending original message 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 03:04:17,291 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7f5f25d3 03:04:17,292 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4cdfae46 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4220e1e8, splicing queue 03:04:17,292 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4cdfae46 queue spliced 0 messages 03:04:17,292 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.034 ms 03:04:17,292 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 ready 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 to ready 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating canCommit 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 validated 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed canCommit 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 ready 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 to ready 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating canCommit 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 validated 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:17,293 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed canCommit 03:04:17,293 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 03:04:17,293 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903692442020, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 03:04:17,293 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 03:04:17,294 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 03:04:17,294 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903692513211, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 03:04:17,294 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 03:04:17,294 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:17,294 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@32d0ffe8 03:04:17,294 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 03:04:17,294 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 03:04:17,294 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@1f645e36 03:04:17,294 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,294 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903699886736} 03:04:17,294 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 03:04:17,294 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@30a771af 03:04:17,294 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 03:04:17,294 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 03:04:17,294 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@e8ec316 03:04:17,294 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,294 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43f9b357 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903700029079} 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating preCommit 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 candidate ready 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - candidate rootPath: / 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - messages: [] 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 prepared 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed preCommit 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating preCommit 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 candidate ready 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - messages: [] 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 prepared 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:17,295 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed preCommit 03:04:17,295 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 03:04:17,295 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903700029079, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 03:04:17,296 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:17,296 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 03:04:17,296 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903699886736, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 03:04:17,296 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:17,296 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:04:17,296 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@3b945290 03:04:17,296 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 03:04:17,296 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 2 03:04:17,296 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@3b945290 03:04:17,296 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,296 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903701870646} 03:04:17,296 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 03:04:17,296 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@63c2e3a 03:04:17,296 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 03:04:17,296 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 2 03:04:17,296 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@63c2e3a 03:04:17,296 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,296 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43f9b357 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903701993428} 03:04:17,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating commit 03:04:17,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 03:04:17,296 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 submitted to persistence 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating commit 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 03:04:17,297 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 submitted to persistence 03:04:17,297 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:17,297 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=180} 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=184} 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:17,298 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 03:04:17,298 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:17,298 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 03:04:17,298 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 03:04:17,298 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:17,298 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 03:04:17,298 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 03:04:17,298 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 03:04:17,298 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@75ece967}, purged=MutableUnsignedLongSet{size=0}} 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1253568983] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 03:04:17,299 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1253568983] 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,299 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3cd74530}, purged=MutableUnsignedLongSet{size=0}} 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: commit - successfulFromPrevious: [] 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-539057888] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 03:04:17,299 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 110.5 μs 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:17,299 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1253568983] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:17,299 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-539057888] 03:04:17,299 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 03:04:17,300 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:17,300 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-539057888] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:17,300 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 127.2 μs 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903701870646, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@378e6b23 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 03:04:17,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 03:04:17,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@3fc175b7 03:04:17,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 03:04:17,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 03:04:17,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED}: initiating purge 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} allocated sequence 3 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@39b6ae21 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,301 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903707299710} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903701993428, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2a5d5fa1 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@82a8acf 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED}: initiating purge 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} allocated sequence 3 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@4d401583 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,301 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43f9b357 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903707531597} 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@19cddbfa}, purged=MutableUnsignedLongSet{size=0}} 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-187787996] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,302 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-187787996] 03:04:17,302 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@59539d38}, purged=MutableUnsignedLongSet{size=0}} 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#131654895] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 03:04:17,302 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 58.99 μs 03:04:17,302 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-187787996] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:17,302 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 03:04:17,302 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#131654895] 03:04:17,302 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:17,302 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 34.35 μs 03:04:17,302 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#131654895] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:17,302 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,303 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,303 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,304 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,304 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,304 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 03:04:17,304 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 03:04:17,304 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,304 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:17,304 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 03:04:17,304 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 03:04:17,304 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,304 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:17,304 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 03:04:17,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 03:04:17,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:04:17,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:17,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 03:04:17,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 03:04:17,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:04:17,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:17,305 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 03:04:17,305 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 03:04:17,305 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@898585d 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 03:04:17,313 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 03:04:17,316 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 03:04:17,316 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903707531597, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 03:04:17,316 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE}: purge completed 03:04:17,316 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} 03:04:17,316 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 03:04:17,316 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2a5d5fa1 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 03:04:17,355 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,405 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,444 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 03:04:17,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 03:04:17,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:04:17,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:17,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 03:04:17,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 03:04:17,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:04:17,445 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@2582250f 03:04:17,446 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:17,446 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 03:04:17,447 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 03:04:17,447 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=903707299710, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 03:04:17,447 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE}: purge completed 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 03:04:17,447 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@365adfc0 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} 03:04:17,447 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:17,447 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@378e6b23 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 03:04:17,447 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 03:04:17,455 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 03:04:17,456 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,456 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,456 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 03:04:17,456 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998] 03:04:17,456 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998] 03:04:17,456 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:04:17,456 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:04:17,456 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:17,456 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:04:17,456 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} allocated sequence 0 03:04:17,457 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 03:04:17,457 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@d1d0d5b 03:04:17,457 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,457 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903862757350} 03:04:17,457 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}} 03:04:17,457 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message size: 406 bytes, total slices to send: 1 03:04:17,457 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message does not need to be sliced - sending original message 03:04:17,457 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@366db288 03:04:17,457 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,457 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43f9b357 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903863196152} 03:04:17,457 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}} 03:04:17,457 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message size: 400 bytes, total slices to send: 1 03:04:17,457 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message does not need to be sliced - sending original message 03:04:17,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:17,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:17,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 ready 03:04:17,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 to ready 03:04:17,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 03:04:17,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:17,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 03:04:17,458 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 ready 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 to ready 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:17,459 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 03:04:17,460 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 03:04:17,460 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903863196152, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 03:04:17,460 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} canCommit completed 03:04:17,460 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 03:04:17,460 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903862757350, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 03:04:17,460 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} canCommit completed 03:04:17,460 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:04:17,460 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:04:17,460 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} allocated sequence 0 03:04:17,460 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:04:17,460 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} allocated sequence 0 03:04:17,461 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 03:04:17,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:17,461 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,461 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:17,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:17,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,461 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,461 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,461 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,461 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:04:17,461 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:04:17,461 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 0 03:04:17,461 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@4846d0ab 03:04:17,461 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,461 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2}, enqueuedTicks=903867449688} 03:04:17,461 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2}} 03:04:17,462 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message size: 490 bytes, total slices to send: 1 03:04:17,462 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message does not need to be sliced - sending original message 03:04:17,462 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 1 03:04:17,462 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@7c1595f3 03:04:17,462 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2}, enqueuedTicks=903867848618} 03:04:17,462 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=1, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2}} 03:04:17,462 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message size: 491 bytes, total slices to send: 1 03:04:17,462 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message does not need to be sliced - sending original message 03:04:17,462 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 2 03:04:17,462 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:04:17,462 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:04:17,462 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} allocated sequence 0 03:04:17,462 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:04:17,462 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:04:17,462 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} allocated sequence 0 03:04:17,462 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:17,462 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,462 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:17,463 PM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:17,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:17,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,463 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,463 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,463 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} with Envelope{sessionId=1, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 03:04:17,463 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2}, enqueuedTicks=903867449688, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 03:04:17,463 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0} 03:04:17,463 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 03:04:17,463 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2}, enqueuedTicks=903867848618, sessionId=1, txSequence=6} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 03:04:17,463 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1} 03:04:17,463 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,464 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-372: Closing 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:17,465 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-373: Closing 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:17,465 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 03:04:17,465 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:17,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:17,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:17,465 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 03:04:17,465 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 03:04:17,465 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 03:04:17,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:17,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:17,465 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 03:04:17,514 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:17,515 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,515 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:17,515 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@4e300af9 03:04:17,515 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,515 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903921098457} 03:04:17,515 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, message: Envelope{sessionId=1, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}} 03:04:17,515 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message size: 452 bytes, total slices to send: 1 03:04:17,515 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message does not need to be sliced - sending original message 03:04:17,516 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@6e89bb3 03:04:17,516 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,516 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43f9b357 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=903921845245} 03:04:17,516 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE}} 03:04:17,516 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message size: 433 bytes, total slices to send: 1 03:04:17,516 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-369: Message does not need to be sliced - sending original message 03:04:17,516 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:04:17,516 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 03:04:17,516 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:04:17,516 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 03:04:17,516 PM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:04:17,516 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 03:04:17,517 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 03:04:17,518 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=1, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#469705943], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 03:04:17,518 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#469705943], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 03:04:17,518 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=1, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#469705943], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:17,518 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#469705943], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:17,518 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,519 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:04:17,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:04:17,519 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:04:17,519 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:04:17,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 03:04:17,519 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:17,519 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 03:04:17,519 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@497689f1, splicing queue 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@a6d8d27 queue spliced 1 messages 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 1 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43f9b357 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@45e64431, splicing queue 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@43f9b357 queue spliced 1 messages 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 2 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,525 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:17,525 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:17,525 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:17,525 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:17,525 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:17,525 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5fa04a8, attempting to connect 03:04:17,526 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2fb0a59b, attempting to connect 03:04:17,527 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 03:04:17,527 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$c], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 03:04:17,528 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 03:04:17,528 PM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry$d], minVersion=POTASSIUM, maxVersion=POTASSIUM}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 03:04:17,530 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:17,530 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Connect attempt to people failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:17,531 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:17,531 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 2 shard people failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:17,531 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:17,531 PM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:17,531 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:17,532 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 2, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:598) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:369) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:17,568 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:17,568 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:17,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 03:04:17,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 03:04:17,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,568 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 03:04:17,568 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,568 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:04:17,569 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:04:17,569 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:17,569 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:17,569 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:17,569 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 03:04:17,569 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 03:04:17,569 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:17,569 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 03:04:17,569 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:17,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:17,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 03:04:17,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:04:17,573 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 03:04:17,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 03:04:17,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 03:04:17,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:04:17,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 03:04:17,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 03:04:17,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 03:04:17,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:17,575 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:04:17,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 03:04:17,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 03:04:17,575 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:04:17,575 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:17,575 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 03:04:17,575 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:17,575 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 03:04:17,576 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:04:17,578 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:17,578 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:17,579 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 03:04:17,579 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:04:17,579 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:17,579 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:17,579 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 03:04:17,579 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:17,580 PM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 03:04:17,618 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:17,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 03:04:17,618 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998] 03:04:17,618 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 03:04:17,618 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} allocated sequence 1 03:04:17,619 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@28851b30 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 03:04:17,619 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:17,619 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} allocated sequence 1 03:04:17,619 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@1d24196f 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 03:04:17,619 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 skipping transactions [3, 4, 5] 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:17,619 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,619 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:17,619 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:17,619 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 03:04:17,619 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 03:04:17,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:17,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 03:04:17,620 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 03:04:17,621 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 03:04:17,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 03:04:17,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 03:04:17,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,621 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 03:04:17,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 03:04:17,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 03:04:17,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 03:04:17,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:17,622 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 03:04:17,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 03:04:17,622 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:04:17,622 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 03:04:17,622 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:17,622 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 03:04:17,622 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 03:04:17,623 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 03:04:17,623 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:04:17,624 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:17,624 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:17,624 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=13] 03:04:17,624 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:17,624 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:17,624 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 03:04:17,624 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 03:04:17,625 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 03:04:17,625 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:17,626 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 03:04:17,688 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 106, lastApplied : 1, commitIndex : 1 03:04:17,738 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 112, lastApplied : 1, commitIndex : 1 03:04:17,808 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 03:04:17,857 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 116, lastApplied : 1, commitIndex : 1 03:04:17,928 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 03:04:17,977 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 03:04:18,15 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:04:18,48 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 03:04:18,98 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 03:04:18,168 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 03:04:18,217 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 117, lastApplied : 1, commitIndex : 1 03:04:18,288 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 03:04:18,337 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 03:04:18,408 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 03:04:18,458 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 03:04:18,528 PM [cluster-test-shard-dispatcher-28] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 03:04:18,546 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,546 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:18,547 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:18,547 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:18,547 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:18,547 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,547 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@e6c63b5] 03:04:18,547 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:04:18,547 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:04:18,547 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:18,547 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:18,547 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@e6c63b5] 03:04:18,547 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@76cebc89] 03:04:18,547 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@655b564f, attempting to connect 03:04:18,547 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@76cebc89] 03:04:18,547 PM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:18,547 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@49183566, attempting to connect 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@365adfc0 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} previous state FLUSHED 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} previous state SEALED 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} previous state SEALED 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} previous state SEALED 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 03:04:18,548 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 03:04:18,549 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} 03:04:18,549 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 0 03:04:18,549 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2110/0x00000008016cd400@4f7b4dd3 03:04:18,549 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:18,549 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true}, enqueuedTicks=903921098457} 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 replayed 1 successful requests 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 1 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@28851b30 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904024733783} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 0 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating preCommit 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@4e300af9 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true}, enqueuedTicks=903921098457} 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 candidate ready 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - candidate rootPath: / 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - messages: [] 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=2} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 prepared 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 replayed 2 successful requests 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed preCommit 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating canCommit 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 03:04:18,550 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 validated 03:04:18,550 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 03:04:18,551 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1} 03:04:18,551 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed canCommit 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@367416a3 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3833e736 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@21d0ff85, splicing queue 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3833e736 queue spliced 0 messages 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 3.131 ms 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true}, enqueuedTicks=903921098457, sessionId=2, txSequence=0} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 03:04:18,551 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904024733783, sessionId=2, txSequence=1} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true}, enqueuedTicks=903921098457, sessionId=2, txSequence=2} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} canCommit completed 03:04:18,552 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2b1a6971 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} previous state FLUSHED 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} previous state FLUSHED 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2b1a6971 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 0 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2110/0x00000008016cd400@4f7b4dd3 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} scheduling timeout in 30 seconds 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3dcda3bf transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true}, enqueuedTicks=903921845245} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 replayed 1 successful requests 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 1 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@1d24196f 03:04:18,552 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3dcda3bf transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904025270267} 03:04:18,552 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2b1a6971 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], modifications=1, protocol=THREE_PHASE} 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating preCommit 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 0 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@6e89bb3 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3dcda3bf transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true}, enqueuedTicks=903921845245} 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 candidate ready 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - candidate rootPath: / 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6ae8fe2a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1ae0f967 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - messages: [] 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3c6a0169 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@44c3a674, splicing queue 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: preCommit - successfulFromPrevious: [] 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3c6a0169 queue spliced 0 messages 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 prepared 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed preCommit 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 1.454 ms 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating canCommit 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true}, enqueuedTicks=903921845245, sessionId=3, txSequence=0} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 03:04:18,553 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,553 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 validated 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:18,554 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904025270267, sessionId=3, txSequence=1} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed canCommit 03:04:18,554 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:18,554 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 03:04:18,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 2 03:04:18,554 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=true}, enqueuedTicks=903921845245, sessionId=3, txSequence=2} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 03:04:18,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@784fd6ba 03:04:18,554 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} canCommit completed 03:04:18,554 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,554 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,554 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 2 03:04:18,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@784fd6ba 03:04:18,554 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:18,554 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904960070876} 03:04:18,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 2 03:04:18,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@3e18cffb 03:04:18,554 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,554 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,554 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating commit 03:04:18,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 2 03:04:18,554 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@3e18cffb 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,554 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 03:04:18,554 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3dcda3bf transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904960307041} 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 78 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 03:04:18,554 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 submitted to persistence 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating commit 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 185 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:18,555 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}]]] 03:04:18,556 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 submitted to persistence 03:04:18,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 03:04:18,556 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=78} 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}]]] 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=185} 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,557 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 03:04:18,558 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: commit - successfulFromPrevious: [] 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: commit - successfulFromPrevious: [] 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:04:18,558 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 03:04:18,558 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 03:04:18,559 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904960070876, sessionId=2, txSequence=3} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} doCommit completed 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@365adfc0 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 03:04:18,559 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 3 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@14432508 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED}: initiating purge 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} allocated sequence 3 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@975cd26 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904964789384} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 32.36 μs 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904960307041, sessionId=3, txSequence=3} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} doCommit completed 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: initiating purge 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 3 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@65f8e428 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED}: initiating purge 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 116.2 μs 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} allocated sequence 3 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@3c510ee5 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 03:04:18,559 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3dcda3bf transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904965058240} 03:04:18,559 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 1 03:04:18,559 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@3ec40f93 03:04:18,559 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,559 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,559 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 03:04:18,559 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,559 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 03:04:18,559 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,559 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:18,559 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 1 03:04:18,559 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@3ec40f93 03:04:18,559 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904965434060} 03:04:18,559 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,559 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 1 03:04:18,559 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@3169645 03:04:18,559 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,559 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,559 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,559 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 1 03:04:18,559 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@3169645 03:04:18,560 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3dcda3bf transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904965683586} 03:04:18,559 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating preCommit 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 candidate ready 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - candidate rootPath: / 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating preCommit 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - messages: [] 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: preCommit - successfulFromPrevious: [] 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 prepared 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 candidate ready 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - candidate rootPath: / 03:04:18,560 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed preCommit 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - messages: [] 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 03:04:18,560 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 prepared 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:18,560 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904965683586, sessionId=3, txSequence=5} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 03:04:18,560 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed preCommit 03:04:18,560 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:18,560 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 03:04:18,560 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904965434060, sessionId=2, txSequence=5} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 03:04:18,560 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:18,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 2 03:04:18,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@56a7a9b 03:04:18,561 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,561 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,561 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 2 03:04:18,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@56a7a9b 03:04:18,561 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904966893896} 03:04:18,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 2 03:04:18,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@7df50ada 03:04:18,561 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating commit 03:04:18,561 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,561 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 2 03:04:18,561 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@7df50ada 03:04:18,561 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3dcda3bf transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904967102401} 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating commit 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 223 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:18,561 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 submitted to persistence 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}]]] 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 submitted to persistence 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 03:04:18,561 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,562 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 03:04:18,562 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 03:04:18,562 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 03:04:18,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 20.14 μs 03:04:18,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,563 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 55.36 μs 03:04:18,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 03:04:18,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 03:04:18,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:04:18,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 03:04:18,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 03:04:18,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 03:04:18,563 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}], SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}]]] 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=223} 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 03:04:18,564 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@29e32a23 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@72d265e4 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: commit - successfulFromPrevious: [] 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 03:04:18,564 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,564 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 03:04:18,565 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:18,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:04:18,565 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,565 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: commit - successfulFromPrevious: [] 03:04:18,565 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 03:04:18,565 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 03:04:18,565 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 03:04:18,565 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 03:04:18,565 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904964789384, sessionId=2, txSequence=4} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE}: purge completed 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@365adfc0 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904966893896, sessionId=2, txSequence=6} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} doCommit completed 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@365adfc0 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: initiating purge 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 3 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@4e7278a9 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 69.92 μs 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 36.69 μs 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED}: initiating purge 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} allocated sequence 3 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@bd91bb0 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904971179833} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904965058240, sessionId=3, txSequence=4} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE}: purge completed 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2b1a6971 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: purge completed 03:04:18,565 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 03:04:18,565 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904967102401, sessionId=3, txSequence=6} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 03:04:18,565 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} doCommit completed 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 03:04:18,565 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: initiating purge 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 3 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@7f7633bb 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#100640668], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED}: initiating purge 03:04:18,565 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} allocated sequence 3 03:04:18,566 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@73370e19 03:04:18,566 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=3, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 03:04:18,566 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3dcda3bf transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904971568792} 03:04:18,566 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,566 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 03:04:18,566 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 03:04:18,566 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:18,566 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 03:04:18,566 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 03:04:18,567 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} allocated sequence 0 03:04:18,567 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@63d33a60 03:04:18,567 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false}, enqueuedTicks=904973093750} 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 03:04:18,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 03:04:18,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct canCommit 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 03:04:18,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 03:04:18,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 validated 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:18,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct preCommit 03:04:18,567 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 03:04:18,567 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 candidate ready 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - candidate rootPath: / 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - messages: [] 03:04:18,568 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: preCommit - successfulFromPrevious: [] 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 prepared 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct commit 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,568 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,568 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 325 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 03:04:18,568 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:18,568 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 03:04:18,568 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,568 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 submitted to persistence 03:04:18,568 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 94.67 μs 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 03:04:18,569 PM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 03:04:18,569 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,569 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 39.70 μs 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=325} 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=6, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,570 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 03:04:18,571 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 6, nextIndex: 7 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@63154ce6 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 4 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904971179833, sessionId=2, txSequence=7} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE}: purge completed 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: purge completed 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@365adfc0 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 6 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: commit - successfulFromPrevious: [] 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false} with Envelope{sessionId=2, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 03:04:18,571 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false}, enqueuedTicks=904973093750, sessionId=2, txSequence=8} with Envelope{sessionId=2, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} directCommit completed 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED}: initiating purge 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} allocated sequence 1 03:04:18,571 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@31556ee 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904977462660} 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 03:04:18,571 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} allocated sequence 0 03:04:18,571 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,571 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@54402ce1 03:04:18,571 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,571 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false}, enqueuedTicks=904977638624} 03:04:18,572 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 46.82 μs 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct canCommit 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 validated 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct preCommit 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 candidate ready 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - candidate rootPath: / 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - messages: [] 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: preCommit - successfulFromPrevious: [] 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 prepared 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct commit 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 5 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 6 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 8, leaderSnapShotIndex: 5 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 03:04:18,572 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 submitted to persistence 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 03:04:18,572 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 03:04:18,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:04:18,573 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=5, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 03:04:18,573 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 5, nextIndex: 6 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@67f74872 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,573 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:18,574 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 03:04:18,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,574 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904971568792, sessionId=3, txSequence=7} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 03:04:18,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 03:04:18,574 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE}: purge completed 03:04:18,574 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2b1a6971 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} 03:04:18,574 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: purge completed 03:04:18,574 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@12bfce97 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 03:04:18,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 03:04:18,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 03:04:18,574 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 4 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 6 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=8, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 8 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 6 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 5 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,574 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,574 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 03:04:18,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 8, nextIndex: 9 03:04:18,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 6 03:04:18,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 7 03:04:18,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 03:04:18,574 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@47c17cb1 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,575 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904977462660, sessionId=2, txSequence=9} with Envelope{sessionId=2, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE}: purge completed 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 8 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: commit - successfulFromPrevious: [] 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from COMMIT_PENDING to COMMITTED 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false} with Envelope{sessionId=2, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false}, enqueuedTicks=904977638624, sessionId=2, txSequence=10} with Envelope{sessionId=2, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} directCommit completed 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED}: initiating purge 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} allocated sequence 1 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@6561607d 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904981112321} 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 03:04:18,575 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 83.46 μs 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 03:04:18,575 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:18,576 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,576 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 03:04:18,576 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 03:04:18,576 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 03:04:18,576 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 53.34 μs 03:04:18,576 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} allocated sequence 0 03:04:18,576 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@2a83a50 03:04:18,576 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false}, enqueuedTicks=904982153306} 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct canCommit 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 validated 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct preCommit 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 candidate ready 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - candidate rootPath: / 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - messages: [] 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: preCommit - successfulFromPrevious: [] 03:04:18,576 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 prepared 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct commit 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 10, leaderSnapShotIndex: 7 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 submitted to persistence 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 03:04:18,577 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 03:04:18,579 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 8 03:04:18,579 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 6, lastIndex(): 8 03:04:18,579 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 7 03:04:18,579 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 03:04:18,579 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 8 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,580 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 03:04:18,580 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 03:04:18,580 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=10, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,580 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 215.0 μs 03:04:18,581 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 03:04:18,581 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 03:04:18,581 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 10, nextIndex: 11 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 8 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 9 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@6a2de2a1 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 10 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: commit - successfulFromPrevious: [] 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 03:04:18,582 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904981112321, sessionId=2, txSequence=11} with Envelope{sessionId=2, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE}: purge completed 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false} with Envelope{sessionId=2, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#469705943], coordinated=false}, enqueuedTicks=904982153306, sessionId=2, txSequence=12} with Envelope{sessionId=2, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} directCommit completed 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED}: initiating purge 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} allocated sequence 1 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@487f23c7 03:04:18,583 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@423b605d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904988868384} 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,583 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 108.4 μs 03:04:18,583 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 03:04:18,583 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:18,583 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 10 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 8, lastIndex(): 10 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 9 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,583 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 10 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 03:04:18,583 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 03:04:18,583 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-1922813318] 03:04:18,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,584 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 11 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 03:04:18,584 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=10, prevLogTerm=5, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}]]] 03:04:18,584 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 03:04:18,584 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 03:04:18,584 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 03:04:18,584 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 66.62 μs 03:04:18,584 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-369220039] 03:04:18,585 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 03:04:18,585 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:18,585 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815] 03:04:18,585 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=10, prevLogTerm=5, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}]]] 03:04:18,585 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 03:04:18,585 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 03:04:18,585 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 03:04:18,585 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 03:04:18,585 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 03:04:18,585 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@575e4305: creating a new snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:04:18,585 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=11, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:18,585 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 03:04:18,585 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:18,585 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:18,585 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:18,586 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:18,586 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 03:04:18,586 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:18,586 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:04:18,586 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:18,586 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@2da473b4 03:04:18,586 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:18,586 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:18,586 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 03:04:18,586 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@48b4038c, attempting to connect 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 11, nextIndex: 12 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 11 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 10 to 11 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@685fbccb 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,587 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]} with Envelope{sessionId=2, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1}} 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 10 for term 5 03:04:18,587 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#469705943]}, enqueuedTicks=904988868384, sessionId=2, txSequence=13} with Envelope{sessionId=2, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1}} 03:04:18,587 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE}: purge completed 03:04:18,587 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@e4647e8 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=11 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 03:04:18,587 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=6950, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 11 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 11 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 10 to 11 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 10 for term 5 03:04:18,588 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:18,588 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=11 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@18efb22e instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ded3f57 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ded3f57 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2089/0x00000008016bf9a8@711cc09d 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6cfae087 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=904991884669} 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@575e4305 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@9e141f4 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3dc2c018 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2a4de715, splicing queue 03:04:18,588 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3dc2c018 queue spliced 0 messages 03:04:18,589 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 464.4 μs 03:04:18,590 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 03:04:18,591 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-370: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 03:04:18,591 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-370: Message size: 514 bytes, total slices to send: 1 03:04:18,591 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-370: Message does not need to be sliced - sending original message 03:04:18,592 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 03:04:18,592 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=904991884669, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 03:04:18,592 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0} 03:04:18,592 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 03:04:18,592 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:04:18,592 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:04:18,592 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1908564765] 03:04:18,593 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:18,593 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 03:04:18,593 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 03:04:18,593 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:04:18,593 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@78d665c5 03:04:18,593 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:18,593 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 03:04:18,593 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=13] 03:04:18,593 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@375436e5, attempting to connect 03:04:18,594 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 03:04:18,594 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=7260, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:04:18,595 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3501f598 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1edd533d 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state OPEN 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1edd533d 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2089/0x00000008016bf9a8@64602e40 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bfeea3d transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=904998778662} 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@575e4305 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@71e97bef 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5d99a54 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@602eb681, splicing queue 03:04:18,595 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5d99a54 queue spliced 0 messages 03:04:18,596 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 519.1 μs 03:04:18,596 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 03:04:18,597 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-371: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 03:04:18,597 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-371: Message size: 361 bytes, total slices to send: 1 03:04:18,597 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-371: Message does not need to be sliced - sending original message 03:04:18,597 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 03:04:18,598 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1729389907], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=904998778662, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 03:04:18,598 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0} 03:04:18,598 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:04:18,598 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,598 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:18,598 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 03:04:18,598 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 03:04:18,598 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 03:04:18,598 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:18,598 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,598 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:04:18,598 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 03:04:18,598 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 03:04:18,598 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 03:04:18,599 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 03:04:18,599 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 03:04:18,599 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:18,599 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 03:04:18,599 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:18,599 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:18,603 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:18,603 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:18,605 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:18,605 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:18,605 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:18,606 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:18,608 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:18,621 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 03:04:18,621 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 03:04:18,621 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 03:04:18,621 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-375: Closing 03:04:18,621 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-374: Closing 03:04:18,621 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 03:04:18,621 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 03:04:18,621 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 03:04:18,622 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:18,622 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:19,616 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-374975708589141286] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:19,636 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7678148366176179230] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:19,640 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:19,641 PM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:19,642 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:19.647] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config [WARN] [03/07/2024 15:04:19.665] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:19,683 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:04:19,688 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2363942465317760575] 03:04:19,689 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:19,690 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:19,690 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:19,694 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:19,694 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:19,702 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6c05f734 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=SUCCESSOR} 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 1 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1819958818], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@40edf9e3 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:19,702 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@428c2683 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1819958818]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@4c411e52 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-2032649983], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:19,702 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b0acb84 aborted transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-128796272], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@53c48748 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 03:04:19,702 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@45670808 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-128796272]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@532fa990 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#512762735], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:19,702 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@18efb22e aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1729389907], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@7ca260f7 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 03:04:19,702 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 03:04:19,703 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1729389907], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2107/0x00000008016ccb68@38d542bf 03:04:19,703 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#584216815], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:19,703 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6cfae087 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1729389907], modifications=0, protocol=ABORT}, enqueuedTicks=906108701102} 03:04:19,703 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-366: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1729389907], modifications=0, protocol=ABORT}} 03:04:19,703 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-366: Message size: 292 bytes, total slices to send: 1 03:04:19,703 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-366: Message does not need to be sliced - sending original message 03:04:19,703 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3501f598 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 03:04:19,703 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 03:04:19,703 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1729389907], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@581aa428 03:04:19,703 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 03:04:19,703 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 03:04:19,703 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1729389907], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2107/0x00000008016ccb68@c66f1b8 03:04:19,703 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-802513998], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:19,703 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4bfeea3d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1729389907], modifications=0, protocol=ABORT}, enqueuedTicks=906109217714} 03:04:19,703 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-366: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1729389907], modifications=0, protocol=ABORT}} 03:04:19,703 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-366: Message size: 293 bytes, total slices to send: 1 03:04:19,703 PM [Cleaner-1] [DEBUG] MessageSlicer - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-366: Message does not need to be sliced - sending original message 03:04:19,703 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0} 03:04:19,705 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:19,705 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:19,705 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:19,709 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:19,719 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6885615630491849375] 03:04:19,719 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:19,719 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:19,719 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:19,725 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:19,725 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:19,734 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:19,742 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2445982312096384299] 03:04:19,742 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:19,744 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:19,744 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:19,754 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:19,754 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:19,755 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:19,757 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:19,759 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,770 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,771 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1591654773] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:19,784 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:19,785 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,785 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,785 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:19,785 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,786 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 03:04:19,786 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:04:19,786 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@52f90d33 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@254c3034 03:04:19,786 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@511c19db 03:04:19,786 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709823859786, None)) 03:04:19,786 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823859785,0,0)) 03:04:19,786 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@eb4cc7c 03:04:19,786 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@254c3034 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@269ab7dc 03:04:19,786 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 03:04:19,787 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:19,787 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:19,788 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:19,790 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,792 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:19,792 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,798 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,798 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,798 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894] 03:04:19,798 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 03:04:19,800 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:19,800 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1373184747 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,800 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1373184747] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:19,800 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#372828261] 03:04:19,800 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#372828261] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:19,800 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-222527440] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:19,800 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,800 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 03:04:19,807 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,807 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,807 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 03:04:19,807 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:04:19,807 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@74421a61 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5db92251 03:04:19,807 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@ccb1cbf 03:04:19,807 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:19,807 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,808 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709823859807, None)) 03:04:19,808 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823859806,0,0)) 03:04:19,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@532aad91 03:04:19,808 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5db92251 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@669a2c7b 03:04:19,810 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:19,810 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:19,810 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:04:19,810 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 03:04:19,810 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:04:19,810 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 03:04:19,811 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 03:04:19,811 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,811 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 03:04:19,812 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:19,812 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 03:04:19,813 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:19,813 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,813 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,813 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,813 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632] 03:04:19,813 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 03:04:19,814 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 03:04:19,815 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:19,816 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-429173180] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:19,816 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:19,822 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-429173180 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,827 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-1224212214] 03:04:19,827 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-1224212214] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:19,834 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:19,834 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:19,834 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:04:19,834 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:04:19,834 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 03:04:19,834 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,834 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 03:04:19,834 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 03:04:19,835 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:19,835 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 03:04:19,863 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1591654773] 03:04:19,863 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 03:04:19,863 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894] 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:04:20,100 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 03:04:20,100 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 03:04:20,420 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 03:04:20,420 PM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,420 PM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 03:04:20,437 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:20,710 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:20,710 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:20,711 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:20,711 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,711 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,711 PM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:20,711 PM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,711 PM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,711 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,711 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,721 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:20,721 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,721 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,721 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:20,721 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,722 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,722 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,729 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 03:04:20,729 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,730 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 03:04:20,730 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,730 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:20,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 03:04:20,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:20,731 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:20,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=null 03:04:20,731 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:20,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:20,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,731 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 03:04:20,732 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 03:04:20,732 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:04:20,733 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:20,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:20,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,733 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 03:04:20,733 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:04:20,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:20,733 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:20,733 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:04:20,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 03:04:20,734 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:20,761 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:20,773 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:20,775 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:20,778 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,782 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1984355230] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:20,783 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,785 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 03:04:20,785 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:04:20,785 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2b4a60b1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1a989339 03:04:20,785 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@58760b33 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:20,785 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1709823860785, None)) 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,785 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823860784,0,0)) 03:04:20,785 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3aa4f8f0 03:04:20,785 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1a989339 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@268016fe 03:04:20,785 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 03:04:20,786 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@495bd10d: creating a new snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 03:04:20,786 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 03:04:20,786 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:20,786 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:20,786 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-222527440] 03:04:20,786 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} shard 1 03:04:20,786 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 03:04:20,786 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:20,786 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@9c5ed30 03:04:20,786 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} scheduling timeout in 2 seconds 03:04:20,786 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:20,786 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1269629588] 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1269629588] 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1269629588] 03:04:20,787 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 03:04:20,788 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 03:04:20,788 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1269629588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:20,788 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1846646992 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,788 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1846646992] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:20,789 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-658622140] 03:04:20,790 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#-658622140] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:20,790 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:20,791 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:20,791 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 03:04:20,791 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 03:04:20,791 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:04:20,791 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:04:20,791 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 03:04:20,791 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,791 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,791 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,791 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 03:04:20,792 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:20,792 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 03:04:20,792 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=13] 03:04:20,794 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@adbab19, attempting to connect 03:04:20,803 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 03:04:20,803 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=8370, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:04:20,805 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:20,805 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1a38dfd2 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cb4c36d 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cb4c36d 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2089/0x00000008016bf9a8@69a09568 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1ddb966c transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=907192512961} 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@495bd10d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@41c10c4a 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@ba400b4 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@66c2cf7d, splicing queue 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@ba400b4 queue spliced 0 messages 03:04:20,806 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 803.4 μs 03:04:20,807 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new open snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 03:04:20,808 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-377: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 03:04:20,808 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-377: Message size: 245 bytes, total slices to send: 1 03:04:20,808 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-377: Message does not need to be sliced - sending original message 03:04:20,808 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 03:04:20,809 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=907192512961, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 03:04:20,809 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0} 03:04:20,810 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:20,811 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,811 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-380: Closing 03:04:20,811 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 03:04:20,811 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,812 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:20,812 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:20,817 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:20,817 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:20,817 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:20,817 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:20,819 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:20,830 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 03:04:20,831 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,831 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@495bd10d: creating a new transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:04:20,832 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:04:20,832 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:20,832 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@495bd10d readied transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 03:04:20,832 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@55b01756 03:04:20,832 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:20,832 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1ddb966c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], modifications=1, protocol=SIMPLE}, enqueuedTicks=907237984026} 03:04:20,832 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-378: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], modifications=1, protocol=SIMPLE}} 03:04:20,832 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-378: Message size: 407 bytes, total slices to send: 1 03:04:20,832 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-378: Message does not need to be sliced - sending original message 03:04:20,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,833 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:04:20,833 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:20,833 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 03:04:20,833 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 03:04:20,833 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:04:21,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 03:04:21,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 03:04:21,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 03:04:21,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 03:04:21,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:04:21,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,417 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 03:04:21,417 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,418 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 4 03:04:21,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 03:04:21,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 03:04:21,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 03:04:21,418 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:21,418 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 03:04:21,418 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 03:04:21,419 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:21,419 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 03:04:21,419 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:21,419 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 03:04:21,419 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:21,419 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:21,419 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 03:04:21,419 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:21,419 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:21,419 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,419 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:21,419 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1ddb966c superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@530dfe3d, splicing queue 03:04:21,419 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 03:04:21,420 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1ddb966c queue spliced 1 messages 03:04:21,420 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:21,420 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: refreshing backend for shard 1 03:04:21,420 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:21,420 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:21,420 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:21,420 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-222527440] 03:04:21,420 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:21,420 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 03:04:21,421 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:21,421 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:21,421 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 03:04:21,421 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:21,421 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:21,421 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:21,421 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 03:04:21,421 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 03:04:21,422 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 03:04:21,422 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:04:21,422 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized - notifying 1 OnShardInitialized callbacks 03:04:21,422 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:21,422 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 03:04:21,422 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready - notifying 1 OnShardInitialized callbacks 03:04:21,422 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@40e38dd8] 03:04:21,422 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:04:21,422 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:21,422 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@40e38dd8] 03:04:21,422 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@27caf50e, attempting to connect 03:04:21,423 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 03:04:21,423 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=12290, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:04:21,423 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:21,423 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:21,423 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:21,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cb4c36d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27eb4d3f 03:04:21,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:21,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 03:04:21,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27eb4d3f 03:04:21,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 03:04:21,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], snapshotOnly=true} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@6e01a6d4 to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 replayed 1 successful requests 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27eb4d3f 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], modifications=1, protocol=SIMPLE} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} allocated sequence 0 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@55b01756 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@450bba00 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], coordinated=false}, enqueuedTicks=907237984026} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@495bd10d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@35422cfb 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@483d0dd9 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2be9350c, splicing queue 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@483d0dd9 queue spliced 0 messages 03:04:21,424 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.357 ms 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new ready transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct canCommit 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 validated 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct preCommit 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 candidate ready 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - candidate rootPath: / 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - messages: [] 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 prepared 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct commit 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 192 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=192}] 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=192}] 03:04:21,425 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:21,426 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:21,426 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:21,426 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 submitted to persistence 03:04:21,426 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=192}] 03:04:21,427 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:21,427 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,427 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 03:04:21,427 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 03:04:21,427 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=13] 03:04:21,427 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:21,427 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:21,427 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 03:04:21,428 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 03:04:21,428 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:21,428 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,428 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=192}]]] 03:04:21,428 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,428 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=192}]]] 03:04:21,428 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 03:04:21,428 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:21,428 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=192} 03:04:21,428 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=192}] 03:04:21,429 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 03:04:21,429 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:21,429 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=192}] 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 0 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data Simple{size=192} 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@75e8c2bd}, purged=MutableUnsignedLongSet{size=0}} 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committed transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: commit - successfulFromPrevious: [] 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#128783096] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 03:04:21,429 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:21,430 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#128783096] 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152], coordinated=false}, enqueuedTicks=907237984026, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 03:04:21,430 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#490275152]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@1cf70017 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#490275152]} 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED}: initiating purge 03:04:21,430 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} allocated sequence 1 03:04:21,430 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#490275152]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@2037f9a2 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 153.2 μs 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#128783096] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:21,430 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@450bba00 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#490275152]}, enqueuedTicks=907836227886} 03:04:21,430 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 03:04:21,431 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: purging transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 03:04:21,431 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 03:04:21,431 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 03:04:21,431 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:21,431 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 03:04:21,436 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,436 PM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 03:04:21,436 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 03:04:21,436 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 03:04:21,436 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,436 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:21,437 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:04:21,437 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 03:04:21,437 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:21,437 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:21,437 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:21,437 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 03:04:21,437 PM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:21,437 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:21,438 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 03:04:21,438 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-383: Closing 03:04:21,438 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 03:04:21,438 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 03:04:21,439 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:21,439 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:21,437 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1800871632] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:21,442 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#746235183] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#746235183] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:21,442 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1984355230] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1984355230] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:21,442 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1984355230] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1984355230] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:21,831 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2363942465317760575] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:22,458 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:22,458 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:22,459 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:22,461 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [6885615630491849375] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/07/2024 15:04:22.461] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:22,470 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:22,470 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:04:22.473] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:22,492 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:22,499 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5510958052948463145] 03:04:22,499 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:22,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:22,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:22,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:22,501 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:22,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:22,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:22,512 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:22,515 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:22,523 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [790474099260583864] 03:04:22,523 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:22,524 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:22,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:22,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:22,525 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:22,540 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:04:22,553 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8039122951861041335] 03:04:22,553 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:22,555 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:22,555 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:22,557 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:22,558 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:22,565 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:22,565 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:22,567 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:22,568 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:22,570 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 03:04:22,577 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#852450591] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,577 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 03:04:22,578 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 03:04:22,578 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 03:04:22,578 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,578 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:04:22,578 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@65f29985 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1551d40e 03:04:22,578 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,579 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:22,579 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,579 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,580 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1df6a321 03:04:22,580 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1709823862579, None)) 03:04:22,580 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823862578,0,0)) 03:04:22,581 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1551d40e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@35dfc8e5 03:04:22,581 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 03:04:22,581 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:22,583 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7a777b62 03:04:22,583 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:22,583 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:22,583 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,586 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 03:04:22,587 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,587 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,587 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467] 03:04:22,587 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,587 PM [cluster-test-shard-dispatcher-16] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,587 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,587 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 03:04:22,587 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659] 03:04:22,588 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 03:04:22,588 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 03:04:22,588 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 03:04:22,588 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,588 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,588 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#513660404 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,589 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-2026110588] 03:04:22,589 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#513660404] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,590 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 03:04:22,591 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:22,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:22,591 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:22,591 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:22,591 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:22,591 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 03:04:22,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:22,591 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 03:04:22,591 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 03:04:22,591 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 03:04:22,591 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 03:04:22,591 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 03:04:22,592 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-2026110588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,592 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-584879540 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,592 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 03:04:22,592 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-584879540] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,593 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2102474147] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,593 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-98888070] 03:04:22,593 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-98888070] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,594 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 03:04:22,594 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 03:04:22,594 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 03:04:22,594 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 03:04:22,594 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 03:04:22,595 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 03:04:22,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 03:04:22,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:04:22,595 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3854dc80 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3ea4d234 03:04:22,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1c931732 03:04:22,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1709823862595, None)) 03:04:22,595 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823862594,0,0)) 03:04:22,595 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:22,595 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,595 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,595 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@54fb6943 03:04:22,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3ea4d234 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3c1a007d 03:04:22,595 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#-1296115226] 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-2-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#870559151] 03:04:22,596 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 03:04:22,596 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 03:04:22,597 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 03:04:22,597 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 03:04:22,597 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-98772539 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,597 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#147447020] 03:04:22,597 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-569835045 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,597 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1049605226] 03:04:22,597 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#-1296115226] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,599 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:22,599 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:22,599 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:22,599 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 03:04:22,599 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,599 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 03:04:22,599 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 03:04:22,599 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 03:04:22,599 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 03:04:22,600 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#870559151] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,600 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-98772539] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,600 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#147447020] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,600 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-569835045] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,600 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1049605226] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:22,600 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 03:04:22,601 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:22,601 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:22,601 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:22,601 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 03:04:22,601 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 03:04:22,601 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,601 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 03:04:22,601 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 03:04:22,601 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 03:04:22,601 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 03:04:22,612 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:22,614 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:22,646 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#852450591] 03:04:22,646 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 03:04:22,646 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467] 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 03:04:22,836 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 03:04:22,836 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:04:22,885 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 03:04:22,886 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 03:04:22,886 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 03:04:23,105 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:04:23,106 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,106 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 03:04:23,124 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:23,175 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:04:23,175 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,175 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 03:04:23,204 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:23,415 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:04:23,415 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,415 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 03:04:23,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:04:23,425 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,425 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 03:04:23,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:23,517 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,517 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,522 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,578 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:23,735 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:04:23,735 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:04:23,735 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,735 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,735 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 03:04:23,735 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 03:04:23,735 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,735 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:23,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 03:04:23,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:23,737 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:23,737 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:23,737 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 03:04:23,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 03:04:23,737 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:23,737 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:23,737 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 03:04:23,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:23,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,737 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:23,737 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,737 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:23,737 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:23,738 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:23,738 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:23,738 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:23,738 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:23,738 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 03:04:23,738 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 03:04:23,738 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:23,738 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:23,738 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:23,738 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:23,738 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:04:23,738 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:04:23,738 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 03:04:23,739 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 03:04:23,739 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 03:04:23,739 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 03:04:23,739 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 03:04:23,739 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 03:04:23,739 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,739 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 03:04:23,740 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 03:04:23,740 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:04:23,740 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:23,740 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:04:23,740 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:23,804 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#852450591] 03:04:23,804 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 03:04:23,804 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659] 03:04:23,804 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5eb82a46: creating a new transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:04:23,805 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 03:04:23,805 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:23,805 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:23,805 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2102474147] 03:04:23,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:23,805 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 03:04:23,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 03:04:23,805 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:04:23,805 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:23,805 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 03:04:23,805 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:04:23,805 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 03:04:23,805 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:04:23,805 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2102474147] 03:04:23,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:23,805 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 03:04:23,805 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:04:23,805 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:04:23,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 03:04:23,805 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5eb82a46 readied transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:04:23,806 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@3ecffcd1 03:04:23,806 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:23,806 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@4d458dd9 03:04:23,806 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 03:04:23,806 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1ee303b8, attempting to connect 03:04:23,806 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=13] 03:04:23,807 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6150e2e2, attempting to connect 03:04:23,807 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 03:04:23,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=16050, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:04:23,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 03:04:23,808 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=11990, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:04:23,809 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:23,810 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@115d9baf instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@74beeb50 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@74beeb50 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@20862862 03:04:23,815 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@c8f063d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=910211692637} 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-387: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE}} 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-387: Message size: 395 bytes, total slices to send: 1 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-387: Message does not need to be sliced - sending original message 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5eb82a46 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@43c69d24 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@e523660 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@18b12378, splicing queue 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@e523660 queue spliced 0 messages 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.088 ms 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1bc785b8 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b94196 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b94196 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,816 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@1582089a 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@15c4a02 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=910211801030} 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-387: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE}} 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-387: Message size: 400 bytes, total slices to send: 1 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-387: Message does not need to be sliced - sending original message 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5eb82a46 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3a943311 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@385e975 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@24c6dbd9, splicing queue 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@385e975 queue spliced 0 messages 03:04:23,817 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 636.4 μs 03:04:23,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,817 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 03:04:23,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 03:04:23,817 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 03:04:23,818 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,818 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 03:04:23,818 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:23,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,818 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 03:04:23,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 03:04:23,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 03:04:23,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 03:04:23,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 03:04:23,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:23,818 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 03:04:23,821 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 03:04:23,822 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=910211692637, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 03:04:23,822 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 03:04:23,822 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 03:04:23,822 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=910211801030, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 03:04:23,822 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 03:04:23,822 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:23,822 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@2050567a 03:04:23,822 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 03:04:23,822 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 03:04:23,822 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@3fa5b22c 03:04:23,822 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:23,822 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@c8f063d transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910227911192} 03:04:23,822 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 03:04:23,822 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@7e4df87d 03:04:23,822 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 03:04:23,822 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 03:04:23,822 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@1f879928 03:04:23,822 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:23,822 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@15c4a02 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910228090367} 03:04:23,822 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 03:04:23,822 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:23,823 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 03:04:23,824 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 03:04:23,824 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910227911192, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 03:04:23,824 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:23,824 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 03:04:23,824 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910228090367, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 03:04:23,824 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:04:23,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:04:23,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@3d46ef24 03:04:23,824 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 03:04:23,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 03:04:23,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@3d46ef24 03:04:23,824 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:23,824 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@c8f063d transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910230170599} 03:04:23,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 03:04:23,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@2fe01e72 03:04:23,824 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 03:04:23,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 03:04:23,824 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@2fe01e72 03:04:23,824 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:23,824 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@15c4a02 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910230314923} 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 03:04:23,825 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 03:04:23,826 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 03:04:23,826 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 03:04:23,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 03:04:23,826 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:04:23,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:04:23,826 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:23,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:23,826 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=184} 03:04:23,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 03:04:23,826 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:23,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:23,826 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 03:04:23,826 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 03:04:23,827 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:23,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:23,827 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:23,827 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 03:04:23,827 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:23,827 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 03:04:23,827 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:04:23,827 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:04:23,827 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:23,827 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6b98eb56}, purged=MutableUnsignedLongSet{size=0}} 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@52277988}, purged=MutableUnsignedLongSet{size=0}} 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1317447812] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-955141323] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 03:04:23,829 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:04:23,829 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-955141323] 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1317447812] 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 78.54 μs 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-955141323] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 138.9 μs 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1317447812] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910230170599, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 03:04:23,830 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@115d9baf completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@59e60ce3 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@4271b075 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@c8f063d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910236688932} 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910230314923, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1bc785b8 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@6e93270f 03:04:23,831 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5eb82a46: creating a new snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@7c993117 03:04:23,831 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:23,831 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:23,831 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@15c4a02 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910236990959} 03:04:23,831 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@3e7f85e3 03:04:23,831 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@c8f063d transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=910237190544} 03:04:23,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 03:04:23,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 03:04:23,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:23,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 03:04:23,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 03:04:23,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:23,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 03:04:23,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-385: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 03:04:23,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-385: Message size: 337 bytes, total slices to send: 1 03:04:23,832 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-385: Message does not need to be sliced - sending original message 03:04:23,832 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 03:04:23,832 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 03:04:23,833 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 03:04:23,833 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=910237190544, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 03:04:23,833 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 03:04:23,833 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:04:23,833 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 03:04:23,833 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@1d9215dd 03:04:23,833 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@15c4a02 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=910239464591} 03:04:23,834 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 03:04:23,834 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-386: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 03:04:23,834 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SlicedMessageState - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-386: Message size: 342 bytes, total slices to send: 1 03:04:23,834 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-386: Message does not need to be sliced - sending original message 03:04:23,834 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 03:04:23,834 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-240591260], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=910239464591, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 03:04:23,834 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 03:04:23,835 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,835 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 03:04:23,835 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 03:04:23,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 03:04:23,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 03:04:23,835 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b05fc14 03:04:23,835 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b05fc14 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:04:23,835 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 03:04:23,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@31100fb8 03:04:23,835 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:23,835 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:23,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@31100fb8 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:23,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:04:23,835 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 03:04:23,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:23,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:23,835 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:04:23,835 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards] 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards] 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:23,836 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 03:04:23,836 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 03:04:23,836 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:23,836 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 03:04:23,836 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,836 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 03:04:23,837 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@52f4a593}, purged=MutableUnsignedLongSet{size=0}} 03:04:23,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@509501cb}, purged=MutableUnsignedLongSet{size=0}} 03:04:23,837 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 03:04:23,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 03:04:23,837 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@15c4a02 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@42a25421, splicing queue 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:04:23,837 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1652385896] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 03:04:23,837 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#795546924] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@15c4a02 queue spliced 1 messages 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: refreshing backend for shard 2 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1652385896] 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2102474147] 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@c8f063d superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@324ec233, splicing queue 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@c8f063d queue spliced 1 messages 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: refreshing backend for shard 1 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 105.8 μs 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2102474147] 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1652385896] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:23,837 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#795546924] 03:04:23,838 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:04:23,838 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 50.97 μs 03:04:23,838 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#795546924] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:23,838 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:04:23,838 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 03:04:23,838 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:23,838 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:23,838 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:23,838 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:04:23,838 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 03:04:23,838 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: replication of member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@1edad99c 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: replication of member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@3e5530ae 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testReadWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leader transfer complete 03:04:23,839 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:04:23,839 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} not found in inflight queue, checking pending queue 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910236990959, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72} 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72} 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b94196 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1bc785b8 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-240591260]}, enqueuedTicks=910236688932, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 03:04:23,840 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE}: purge completed 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@74beeb50 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:04:23,840 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@115d9baf purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:04:23,840 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:04:24,245 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 03:04:24,245 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:24,245 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:24,245 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-390: Closing 03:04:24,245 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 03:04:24,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-391: Closing 03:04:24,246 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:04:24,246 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:04:24,246 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 03:04:24,246 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 03:04:24,246 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 03:04:24,246 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 03:04:24,246 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 03:04:24,246 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 03:04:24,247 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#852450591] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#852450591] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:24,248 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:24,248 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:24,252 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:24,253 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:24,253 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:24,253 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:24,267 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 03:04:24,268 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 03:04:24,268 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 03:04:24,268 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 03:04:24,268 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 03:04:24,268 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:04:24,268 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:04:24,268 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 03:04:24,269 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:24,269 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:25,267 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5510958052948463145] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:25,285 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:25,285 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:25,286 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:25.287] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:25,287 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [790474099260583864] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:25,298 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:25,298 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:25,301 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:25.302] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:25,317 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:25,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5470298857962484353] 03:04:25,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:25,324 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:25,324 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:25,330 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:25,330 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:25,336 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:25,336 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:25,336 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:25,338 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:25,345 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4772859771974005948] 03:04:25,345 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:25,345 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:25,345 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:25,356 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:25,356 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:25,360 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1a38dfd2 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#490275152], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@68e947b3 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 03:04:25,360 PM [Cleaner-1] [DEBUG] RemoteProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} has a successor, which should receive seal through a separate request 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#490275152], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2107/0x00000008016ccb68@41c62a37 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#871292894], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:25,360 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@27eb4d3f aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@55f595d7 03:04:25,360 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@450bba00 transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#490275152]}, enqueuedTicks=911766159963} 03:04:25,360 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0} 03:04:25,360 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@74beeb50 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 1 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@466d8e34 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#1064845467], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:25,360 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b94196 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 1 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-240591260], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@649b9d95 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:25,360 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#820948659], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:04:25,360 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0} 03:04:25,375 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:25,388 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-670469578889591803] 03:04:25,388 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:25,397 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:25,397 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:25,397 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:25,397 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:25,401 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:25,402 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:25,404 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 03:04:25,412 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 03:04:25,414 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-501445367] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:25,416 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 03:04:25,416 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 03:04:25,416 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7db7358b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@219ad43b 03:04:25,416 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4c000807 03:04:25,416 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLargeReadReplySlicing, 0, 1709823865416, None)) 03:04:25,416 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823865415,0,0)) 03:04:25,417 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@219ad43b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6bcdb887 03:04:25,417 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} started 03:04:25,417 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 03:04:25,417 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-1-shard-cars-testLargeReadReplySlicing 03:04:25,417 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:25,417 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:25,418 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:25,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@612d0aba 03:04:25,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:25,418 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLargeReadReplySlicing 03:04:25,419 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:25,421 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 03:04:25,434 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:25,434 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:25,434 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-1-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868] 03:04:25,434 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testLargeReadReplySlicing, persistent : true 03:04:25,434 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:25,435 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#12979158 created and ready for shard:member-1-shard-cars-testLargeReadReplySlicing 03:04:25,435 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-983535288] 03:04:25,435 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#12979158] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:25,435 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-983535288] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:25,435 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLargeReadReplySlicing with journal batch size 1 03:04:25,435 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 03:04:25,436 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1306160903] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:25,438 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:25,439 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:25,439 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:25,439 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:25,440 PM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 03:04:25,440 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 03:04:25,440 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 03:04:25,440 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLargeReadReplySlicing is initialized 03:04:25,440 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 03:04:25,440 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 03:04:25,442 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:25,442 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:25,443 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from null to Follower 03:04:25,449 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 03:04:25,449 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-2-shard-cars-testLargeReadReplySlicing 03:04:25,449 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 03:04:25,449 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 03:04:25,449 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@642ccfc6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@69c5e67f 03:04:25,449 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@732d9ce4 03:04:25,450 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLargeReadReplySlicing, 0, 1709823865449, None)) 03:04:25,450 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823865448,0,0)) 03:04:25,450 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:25,450 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:25,450 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7198dd9c 03:04:25,450 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@69c5e67f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5bc04dd5 03:04:25,450 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} started 03:04:25,450 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:25,450 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLargeReadReplySlicing 03:04:25,451 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:25,451 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 03:04:25,451 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-2-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#-1347379825] 03:04:25,451 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testLargeReadReplySlicing, persistent : true 03:04:25,451 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#-1347379825] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:25,451 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#-1160314791 created and ready for shard:member-2-shard-cars-testLargeReadReplySlicing 03:04:25,451 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#-1611640521] 03:04:25,451 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#-1160314791] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:25,451 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#-1611640521] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:25,451 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLargeReadReplySlicing with journal batch size 1 03:04:25,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:25,456 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:25,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 03:04:25,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLargeReadReplySlicing is no longer the leader 03:04:25,456 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:25,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 03:04:25,456 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLargeReadReplySlicing is initialized 03:04:25,457 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 03:04:25,457 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 03:04:25,457 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-2-shard-cars-testLargeReadReplySlicing from null to Follower 03:04:25,500 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-501445367] 03:04:25,501 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: findLocalShard : cars 03:04:25,501 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868] 03:04:25,683 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:25,683 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Received ElectionTimeout - switching to Candidate 03:04:25,683 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLargeReadReplySlicing] 03:04:25,683 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=1, votedFor=member-1-shard-cars-testLargeReadReplySlicing 03:04:25,683 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 1 03:04:25,684 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:25,684 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing (Follower): Closing 03:04:25,684 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 03:04:25,684 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 03:04:25,684 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Follower to Candidate 03:04:25,684 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Follower to Candidate 03:04:25,967 PM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:26,4 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 03:04:26,4 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=2, votedFor=member-1-shard-cars-testLargeReadReplySlicing 03:04:26,4 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 2 03:04:26,283 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 03:04:26,283 PM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=3, votedFor=member-1-shard-cars-testLargeReadReplySlicing 03:04:26,284 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 3 03:04:26,344 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:26,344 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:26,345 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:26,345 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 03:04:26,345 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:26,345 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:26,345 PM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 03:04:26,345 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:26,345 PM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 03:04:26,385 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:26,385 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 03:04:26,385 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 03:04:26,385 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:26,385 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 03:04:26,385 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 03:04:26,401 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:26,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 03:04:26,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 03:04:26,554 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 4 03:04:26,554 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLargeReadReplySlicing 03:04:26,555 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:26,556 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Cluster state: [] 03:04:26,556 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:26,556 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:26,556 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=null 03:04:26,556 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:26,556 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 03:04:26,556 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Election: Leader has following peers: [member-2-shard-cars-testLargeReadReplySlicing] 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: true, hasLeader : true 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: became leader with frontend state for [] 03:04:26,557 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Candidate to Leader 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=13] 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Candidate to Leader 03:04:26,557 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 03:04:26,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing: Closing 03:04:26,558 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: getLeaderAddress leaderId = member-1-shard-cars-testLargeReadReplySlicing peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 03:04:26,558 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done false 03:04:26,558 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLargeReadReplySlicing updated: matchIndex: -1, nextIndex: 0 03:04:26,558 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:26,559 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=13] 03:04:26,559 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:26,559 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:26,559 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 03:04:26,607 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4e6da366: creating a new transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 03:04:26,607 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 03:04:26,607 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:26,607 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:26,607 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1306160903] 03:04:26,607 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:26,607 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=0} shard 1 03:04:26,607 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 03:04:26,607 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:26,607 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=13] 03:04:26,607 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 03:04:26,607 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@557539ef 03:04:26,608 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:26,608 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1186768436], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@79140a1e 03:04:26,608 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=13] 03:04:26,608 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@f00327, attempting to connect 03:04:26,609 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} is not yet known 03:04:26,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, nanosAgo=13470, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 03:04:26,611 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@61e70212 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60cfd666 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60cfd666 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=1} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1186768436], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2088/0x00000008016bf770@43f0dd0c 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@23cd2e25 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=1}, enqueuedTicks=913013756877} 03:04:26,611 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=1}} 03:04:26,612 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: Message size: 449 bytes, total slices to send: 5 03:04:26,612 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: getNextSlice: total size: 449, offset: 0, size: 100, index: 1 03:04:26,628 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/$a#1186768436]] 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1186768436], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2089/0x00000008016bf9a8@323fdbf 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@23cd2e25 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1186768436], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=913013756877} 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@23cd2e25 cannot transmit request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1186768436], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} - delaying it 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4e6da366 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@420ca0c5 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6796c1e8 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@71569473, splicing queue 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6796c1e8 queue spliced 0 messages 03:04:26,629 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 18.19 ms 03:04:26,636 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,638 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] - creating AssembledMessageState 03:04:26,640 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 03:04:26,640 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] - expecting more 03:04:26,646 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,647 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: getNextSlice: total size: 449, offset: 100, size: 100, index: 2 03:04:26,647 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test/user/$a#1186768436]] 03:04:26,648 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,648 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1085826726, lastSliceHashCode: -1085826726 03:04:26,648 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] - expecting more 03:04:26,649 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,649 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: getNextSlice: total size: 449, offset: 200, size: 100, index: 3 03:04:26,649 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test/user/$a#1186768436]] 03:04:26,650 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,650 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: -840689321, lastSliceHashCode: -840689321 03:04:26,650 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] - expecting more 03:04:26,651 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,651 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: getNextSlice: total size: 449, offset: 300, size: 100, index: 4 03:04:26,651 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=-1604171807, replyTo=Actor[akka://cluster-test/user/$a#1186768436]] 03:04:26,652 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=-1604171807, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,652 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -1604171807, lastSliceHashCode: -1604171807 03:04:26,652 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] - expecting more 03:04:26,652 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,652 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: getNextSlice: total size: 449, offset: 400, size: 49, index: 5 03:04:26,652 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=49, sliceIndex=5, totalSlices=5, lastSliceHashCode=32166009, replyTo=Actor[akka://cluster-test/user/$a#1186768436]] 03:04:26,653 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], data.length=49, sliceIndex=5, totalSlices=5, lastSliceHashCode=32166009, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,653 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex: 5, lastSliceIndex: 4, assembledSize: 400, sliceHashCode: 32166009, lastSliceHashCode: 32166009 03:04:26,653 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] 03:04:26,653 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] 03:04:26,654 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] was removed from the cache due to EXPLICIT 03:04:26,654 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436], modifications=1}} 03:04:26,654 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testLargeReadReplySlicing: allocating new open transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 03:04:26,655 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57], sliceIndex=5, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,655 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] 03:04:26,655 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] 03:04:26,655 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-394: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=394, messageId=57] was removed from the cache due to EXPLICIT 03:04:26,655 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=1} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 03:04:26,655 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@23cd2e25 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1186768436], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=913013756877} 03:04:26,656 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@23cd2e25 transmitted 1 requests 03:04:26,656 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=1}, enqueuedTicks=913013756877, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 03:04:26,656 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0} 03:04:26,657 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 03:04:26,657 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: Message size: 391 bytes, total slices to send: 4 03:04:26,657 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: getNextSlice: total size: 391, offset: 0, size: 100, index: 1 03:04:26,657 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,658 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,658 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] - creating AssembledMessageState 03:04:26,658 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 03:04:26,658 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] - expecting more 03:04:26,659 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-396: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,659 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,659 PM [cluster-test-shard-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: getNextSlice: total size: 391, offset: 100, size: 100, index: 2 03:04:26,659 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,660 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,660 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1569038920, lastSliceHashCode: -1569038920 03:04:26,660 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] - expecting more 03:04:26,661 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-396: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,661 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,661 PM [cluster-test-shard-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: getNextSlice: total size: 391, offset: 200, size: 100, index: 3 03:04:26,661 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,661 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,661 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: 1439492931, lastSliceHashCode: 1439492931 03:04:26,661 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] - expecting more 03:04:26,664 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-396: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,664 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,664 PM [cluster-test-shard-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: getNextSlice: total size: 391, offset: 300, size: 91, index: 4 03:04:26,664 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], data.length=91, sliceIndex=4, totalSlices=4, lastSliceHashCode=-31040526, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], data.length=91, sliceIndex=4, totalSlices=4, lastSliceHashCode=-31040526, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] 03:04:26,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -31040526, lastSliceHashCode: -31040526 03:04:26,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] 03:04:26,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] 03:04:26,666 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] was removed from the cache due to EXPLICIT 03:04:26,666 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 03:04:26,666 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1186768436], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 03:04:26,666 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1186768436], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=913013756877, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 03:04:26,666 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1} 03:04:26,666 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 03:04:26,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-396: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1186768436]] 03:04:26,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] 03:04:26,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] 03:04:26,666 PM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-393: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=393, messageId=58] was removed from the cache due to EXPLICIT 03:04:26,667 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Issuing gracefulStop to shard member-1-shard-cars-testLargeReadReplySlicing 03:04:26,667 PM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLargeReadReplySlicing - waiting on 1 shards 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onShutDown 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Initiating leader transfer 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5596356f 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testLargeReadReplySlicing: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5596356f 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Attempting to transfer leadership 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=13] 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:26,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:26,668 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLargeReadReplySlicing] 03:04:26,668 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=13] 03:04:26,668 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:26,668 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done true 03:04:26,668 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:26,669 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@23cd2e25 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@25fbaeeb, splicing queue 03:04:26,669 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@23cd2e25 queue spliced 0 messages 03:04:26,669 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:26,669 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: refreshing backend for shard 1 03:04:26,669 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:26,669 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLargeReadReplySlicing, matchIndex: -1, lastIndex: -1, isVoting: true 03:04:26,669 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Follower's log matches - sending ElectionTimeout 03:04:26,669 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:26,669 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:26,669 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:26,669 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-1306160903] 03:04:26,669 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Leader transfer complete 03:04:26,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:26,669 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLargeReadReplySlicing, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:27,84 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: All shards shutdown - sending PoisonPill to self 03:04:27,84 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] ShardManager - shard-manager-testLargeReadReplySlicing: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:27,84 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLargeReadReplySlicing 03:04:27,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-396: Closing 03:04:27,84 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 03:04:27,84 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 03:04:27,85 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 03:04:27,85 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 03:04:27,85 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: shutting down 03:04:27,85 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 03:04:27,85 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 03:04:27,85 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-501445367] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-501445367] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:27,85 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-501445367] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-501445367] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:27,85 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-501445367] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#-501445367] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:27,86 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:27,86 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:27,91 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:27,91 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:27,91 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:27,91 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:27,93 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:27,104 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Closing 03:04:27,105 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLargeReadReplySlicing 03:04:27,105 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing (Follower): Closing 03:04:27,105 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 03:04:27,105 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 03:04:27,105 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:27,105 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:28,100 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5470298857962484353] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:28,118 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:28,118 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:28,120 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:28,120 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4772859771974005948] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [03/07/2024 15:04:28.121] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:28,128 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:28,128 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:28,130 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:28.131] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:28,149 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:28,155 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2835405734656160986] 03:04:28,156 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:28,161 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:28,161 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:28,162 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:28,162 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:28,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:28,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:28,176 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:28,178 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:28,184 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1801327992407967497] 03:04:28,184 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:28,189 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:28,189 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:28,195 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:28,195 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:28,208 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:28,220 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-1130813291192702977] 03:04:28,221 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:28,223 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:28,223 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:28,223 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:28,223 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:28,233 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:28,236 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:28,237 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 03:04:28,244 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 03:04:28,244 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:28,247 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 03:04:28,247 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,247 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:28,247 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,250 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 03:04:28,250 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:04:28,250 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6eba945f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@607d67f8 03:04:28,250 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6442ff2f 03:04:28,250 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1709823868250, None)) 03:04:28,250 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823868249,0,0)) 03:04:28,251 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@607d67f8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@10abebfb 03:04:28,251 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@75cdd662 03:04:28,251 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 03:04:28,251 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:28,251 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,251 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:28,253 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:28,255 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 03:04:28,266 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#705386147] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:28,266 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,266 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,266 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722] 03:04:28,267 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 03:04:28,267 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 03:04:28,267 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:28,267 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1885372041 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,267 PM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#2106822168] 03:04:28,267 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1885372041] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:28,267 PM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#2106822168] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:28,268 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 03:04:28,268 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:28,268 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:28,269 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 03:04:28,269 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 03:04:28,269 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 03:04:28,269 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 03:04:28,269 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 03:04:28,270 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 03:04:28,276 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 03:04:28,277 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 03:04:28,277 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 03:04:28,277 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,277 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:04:28,277 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2fcd14af to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6c78887b 03:04:28,283 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:28,284 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@686d6c46 03:04:28,284 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1709823868277, None)) 03:04:28,285 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:28,285 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:28,285 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,285 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823868276,0,0)) 03:04:28,286 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6c78887b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@44c4f006 03:04:28,286 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 03:04:28,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@31455ee1 03:04:28,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:28,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,286 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,286 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#-355441230] 03:04:28,287 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 03:04:28,287 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#-355441230] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:28,287 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-133287344 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,287 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-133287344] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:28,287 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#47078072] 03:04:28,288 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#47078072] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:28,292 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 03:04:28,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:28,293 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:28,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 03:04:28,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 03:04:28,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 03:04:28,293 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 03:04:28,293 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 03:04:28,293 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,293 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 03:04:28,293 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 03:04:28,298 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:28,301 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:28,336 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] 03:04:28,336 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 03:04:28,337 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722] 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 03:04:28,545 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 03:04:28,545 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 03:04:28,794 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 03:04:28,794 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:28,795 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 03:04:28,816 PM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:29,84 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 03:04:29,84 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,84 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 03:04:29,175 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:29,175 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:29,175 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:29,175 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,175 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,175 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:29,175 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,175 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,175 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,180 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:29,181 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,181 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,181 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:29,181 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,181 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,236 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:29,355 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 03:04:29,355 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,355 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 03:04:29,355 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:29,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 03:04:29,356 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:29,357 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:29,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 03:04:29,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:29,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,357 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 03:04:29,358 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 03:04:29,358 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 03:04:29,360 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 03:04:29,360 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,360 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=13] 03:04:29,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:29,360 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:29,360 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 03:04:29,360 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 03:04:29,360 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 03:04:29,360 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:29,393 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 03:04:29,394 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from IDLE to TX_OPEN 03:04:29,394 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 03:04:29,394 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:29,394 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:29,394 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#705386147] 03:04:29,394 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} shard 1 03:04:29,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:29,394 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:29,394 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 03:04:29,394 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:04:29,394 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:29,394 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@163db564 03:04:29,394 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from TX_OPEN to IDLE 03:04:29,394 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 03:04:29,394 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 03:04:29,394 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@7e8a8432 03:04:29,394 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=13] 03:04:29,395 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4630a3b9, attempting to connect 03:04:29,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} is not yet known 03:04:29,396 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=12870, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:04:29,397 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:29,397 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:29,397 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:29,397 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@720ccb4a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 03:04:29,397 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915800159716} 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2043/0x00000008016a3198@15d2b26e 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=2, protocol=READY}, enqueuedTicks=915800461754} 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=2, protocol=READY}} 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 416 bytes, total slices to send: 1 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2102/0x00000008016c6a50@7016fac0 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915800589547} 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}} 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 296 bytes, total slices to send: 1 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,398 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:29,399 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:04:29,399 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@42a8e0b7 03:04:29,399 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2af1c214 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3c9f65a2, splicing queue 03:04:29,399 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2af1c214 queue spliced 0 messages 03:04:29,399 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.256 ms 03:04:29,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 03:04:29,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 03:04:29,399 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:04:29,400 PM [cluster-test-shard-dispatcher-12] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,400 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,400 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,400 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@323b3513 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 ready 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 to ready 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 is already in state READY (READY) 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct canCommit 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 validated 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct preCommit 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 candidate ready 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - messages: [] 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 prepared 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct commit 03:04:29,401 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 submitted to persistence 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 03:04:29,402 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 03:04:29,403 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 03:04:29,403 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=2, protocol=READY}, enqueuedTicks=915800461754, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 03:04:29,403 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0} 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=184} 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 03:04:29,403 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 1, nextIndex: 2 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2048/0x00000008016a84c0@74c044fd 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5ae4404}, purged=MutableUnsignedLongSet{size=0}} 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1876007628] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:29,404 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 03:04:29,404 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1876007628] 03:04:29,405 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,405 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 62.78 μs 03:04:29,405 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1876007628] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:29,405 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 03:04:29,405 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915800159716, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 03:04:29,405 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0} 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 1 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@22094b4}, purged=MutableUnsignedLongSet{size=0}} 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915800589547, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1} 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@5a9021f1 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@5fb8cf4c 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915812087854} 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-247540978] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-247540978] 03:04:29,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-247540978] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:29,406 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 99.22 μs 03:04:29,406 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from IDLE to TX_OPEN 03:04:29,407 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:04:29,407 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 03:04:29,407 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@20b58b3d 03:04:29,407 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915812949065} 03:04:29,407 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}} 03:04:29,407 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 456 bytes, total slices to send: 1 03:04:29,407 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,407 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from TX_OPEN to IDLE 03:04:29,407 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 03:04:29,407 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 1 03:04:29,407 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@504a282 03:04:29,407 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915813362666} 03:04:29,407 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}} 03:04:29,407 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,407 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,408 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,408 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 03:04:29,408 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 03:04:29,408 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,408 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5bbbf1f5 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 ready 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 to ready 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 is already in state READY (READY) 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct canCommit 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 validated 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct preCommit 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 candidate ready 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - messages: [] 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 prepared 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct commit 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,411 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 03:04:29,412 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 03:04:29,412 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 03:04:29,412 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:29,412 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 03:04:29,412 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,412 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 03:04:29,412 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 submitted to persistence 03:04:29,412 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 03:04:29,412 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915812949065, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 03:04:29,412 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0} 03:04:29,412 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 03:04:29,412 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,413 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 03:04:29,413 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@5e3fb189 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:04:29,413 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 03:04:29,413 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,414 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 69.90 μs 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915812087854, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@720ccb4a purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915813362666, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} directCommit completed 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@7e187e56 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915820858092} 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,415 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 03:04:29,415 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 03:04:29,415 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@296e1241 03:04:29,415 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915821149920} 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,415 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 91.71 μs 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 03:04:29,415 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 03:04:29,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 03:04:29,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 03:04:29,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 03:04:29,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,416 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6a0d91e7 03:04:29,416 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 03:04:29,416 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message size: 388 bytes, total slices to send: 1 03:04:29,416 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message does not need to be sliced - sending original message 03:04:29,417 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 03:04:29,417 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 03:04:29,417 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915821149920, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 03:04:29,417 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0} 03:04:29,417 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} 03:04:29,417 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 03:04:29,417 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@771668d6 03:04:29,417 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915823450497} 03:04:29,417 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}} 03:04:29,417 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,417 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,417 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from IDLE to TX_OPEN 03:04:29,418 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:04:29,418 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} allocated sequence 0 03:04:29,418 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@5ac30d51 03:04:29,418 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915823847907} 03:04:29,418 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}} 03:04:29,418 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 456 bytes, total slices to send: 1 03:04:29,418 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,418 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from TX_OPEN to IDLE 03:04:29,418 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 03:04:29,418 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} allocated sequence 1 03:04:29,418 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@1e1c7461 03:04:29,418 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915824139235} 03:04:29,418 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}} 03:04:29,418 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,418 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6a0d91e7 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 ready 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 03:04:29,419 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 to ready 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 is already in state READY (READY) 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct canCommit 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 validated 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct preCommit 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 candidate ready 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - candidate rootPath: / 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - messages: [] 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: preCommit - successfulFromPrevious: [] 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 prepared 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct commit 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 6, leaderSnapShotIndex: 2 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 03:04:29,420 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 03:04:29,420 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 submitted to persistence 03:04:29,420 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915823847907, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 03:04:29,420 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0} 03:04:29,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 3 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 4 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,421 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 03:04:29,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 03:04:29,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 3 03:04:29,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 03:04:29,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 03:04:29,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 03:04:29,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@34893642 03:04:29,421 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2114/0x00000008016ceeb8@7e183021 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: commit - successfulFromPrevious: [] 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 03:04:29,422 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 03:04:29,422 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,422 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 43.56 μs 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 3, lastIndex(): 6 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 03:04:29,422 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 03:04:29,422 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 03:04:29,423 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 03:04:29,423 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 03:04:29,423 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915820858092, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915823450497, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1} 03:04:29,423 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN}: initiating purge 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 2 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@3b1aeff1 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915828795831} 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915824139235, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 03:04:29,423 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} directCommit completed 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED}: initiating purge 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} allocated sequence 2 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@4f279491 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915828944545} 03:04:29,423 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,423 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 03:04:29,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 66.74 μs 03:04:29,423 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 03:04:29,423 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 0 03:04:29,423 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@67c8fc7e 03:04:29,423 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915829332514} 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 03:04:29,424 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,425 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6a54dd66 03:04:29,425 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 03:04:29,425 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message size: 408 bytes, total slices to send: 1 03:04:29,425 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message does not need to be sliced - sending original message 03:04:29,425 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,425 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 03:04:29,426 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 03:04:29,426 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915829332514, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 03:04:29,426 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0} 03:04:29,426 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} 03:04:29,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 1 03:04:29,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@101c3b77 03:04:29,426 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915832101943} 03:04:29,426 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}} 03:04:29,426 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,426 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,426 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from IDLE to TX_OPEN 03:04:29,426 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:04:29,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} allocated sequence 0 03:04:29,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@13019d49 03:04:29,426 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915832419881} 03:04:29,426 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=f, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}} 03:04:29,426 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 456 bytes, total slices to send: 1 03:04:29,426 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,426 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from TX_OPEN to IDLE 03:04:29,426 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 03:04:29,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} allocated sequence 1 03:04:29,426 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@57f11502 03:04:29,427 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915832672467} 03:04:29,427 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=10, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}} 03:04:29,427 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,427 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6a54dd66 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 ready 03:04:29,427 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 to ready 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 is already in state READY (READY) 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct canCommit 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 validated 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct preCommit 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 candidate ready 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - candidate rootPath: / 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - messages: [] 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: preCommit - successfulFromPrevious: [] 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 prepared 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct commit 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 10, leaderSnapShotIndex: 5 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}], SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 submitted to persistence 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,428 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}], SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 03:04:29,429 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,429 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915832419881, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 03:04:29,429 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0} 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,429 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 03:04:29,429 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 10, nextIndex: 11 03:04:29,429 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 6 03:04:29,429 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 03:04:29,429 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@6c189b5b 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 8 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@1ee152dd 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 9 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2114/0x00000008016ceeb8@3af7cda0 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 10 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: commit - successfulFromPrevious: [] 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 03:04:29,430 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 03:04:29,430 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,430 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 51.90 μs 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 10 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 6, lastIndex(): 10 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915828795831, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915828944545, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE}: purge completed 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 8 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915832101943, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN}: initiating purge 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} allocated sequence 2 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 9 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@30c56465 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915837152629} 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 10 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915832672467, sessionId=0, txSequence=16} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} directCommit completed 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED}: initiating purge 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} allocated sequence 2 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@75c7ec19 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915837343254} 03:04:29,431 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,431 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 0 03:04:29,431 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@2e096391 03:04:29,431 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915837556049} 03:04:29,431 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 03:04:29,432 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 03:04:29,431 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,432 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 519.2 μs 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,438 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@759039c7 03:04:29,439 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 03:04:29,439 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,439 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message size: 431 bytes, total slices to send: 1 03:04:29,439 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message does not need to be sliced - sending original message 03:04:29,439 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 03:04:29,440 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 03:04:29,440 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915837556049, sessionId=0, txSequence=19} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 03:04:29,440 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0} 03:04:29,440 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} 03:04:29,440 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 1 03:04:29,440 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@4e16d4c2 03:04:29,440 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915845949849} 03:04:29,440 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=14, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}} 03:04:29,440 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,440 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,440 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from IDLE to TX_OPEN 03:04:29,440 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:04:29,440 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} allocated sequence 0 03:04:29,440 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@e993432 03:04:29,440 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915846360199} 03:04:29,440 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=15, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}} 03:04:29,440 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 456 bytes, total slices to send: 1 03:04:29,440 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,440 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from TX_OPEN to IDLE 03:04:29,440 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 03:04:29,440 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} allocated sequence 1 03:04:29,440 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@6a82b6fd 03:04:29,440 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915846619945} 03:04:29,440 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=16, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}} 03:04:29,441 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,441 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 03:04:29,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 13, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 03:04:29,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,441 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@759039c7 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 ready 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 to ready 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 is already in state READY (READY) 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct canCommit 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 validated 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct preCommit 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 candidate ready 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - candidate rootPath: / 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - messages: [] 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: preCommit - successfulFromPrevious: [] 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 prepared 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct commit 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 14, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 14, leaderSnapShotIndex: 9 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}], SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 submitted to persistence 03:04:29,442 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,443 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 03:04:29,443 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 03:04:29,443 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915846360199, sessionId=0, txSequence=21} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}], SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 03:04:29,443 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0} 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 03:04:29,443 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=14, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 03:04:29,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 03:04:29,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,444 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 14, nextIndex: 15 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 14, lastAppliedIndex: 10 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 11 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@35469d00 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 12 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@3a0e751e 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 13 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2114/0x00000008016ceeb8@5d9a6466 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 14 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: commit - successfulFromPrevious: [] 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from COMMIT_PENDING to COMMITTED 03:04:29,444 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:04:29,445 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 03:04:29,445 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 15, leaderLastIndex: 14, leaderSnapShotIndex: 13 03:04:29,445 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 03:04:29,445 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,445 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 70.94 μs 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 14 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 14, context.getLastApplied(): 10, lastIndex(): 14 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 11 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 12 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 13 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 14 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 03:04:29,445 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 03:04:29,445 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915837152629, sessionId=0, txSequence=17} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE}: purge completed 03:04:29,446 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915837343254, sessionId=0, txSequence=18} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE}: purge completed 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915845949849, sessionId=0, txSequence=20} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN}: initiating purge 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} allocated sequence 2 03:04:29,446 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@27d00013 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915851917988} 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 03:04:29,446 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915846619945, sessionId=0, txSequence=22} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 03:04:29,446 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} directCommit completed 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 84.57 μs 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED}: initiating purge 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} allocated sequence 2 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@44f8393d 03:04:29,446 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915852217795} 03:04:29,447 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 03:04:29,447 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 0 03:04:29,447 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@3c1629cd 03:04:29,447 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915853185929} 03:04:29,447 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 03:04:29,447 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,447 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,447 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 15, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 16, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@4b5aa498 03:04:29,448 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,448 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message size: 451 bytes, total slices to send: 1 03:04:29,448 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message does not need to be sliced - sending original message 03:04:29,448 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 03:04:29,449 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 03:04:29,449 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915853185929, sessionId=0, txSequence=25} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 03:04:29,449 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0} 03:04:29,449 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} 03:04:29,449 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 1 03:04:29,449 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@3578546a 03:04:29,449 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915855226680} 03:04:29,449 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}} 03:04:29,449 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,449 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,449 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from IDLE to TX_OPEN 03:04:29,449 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:04:29,449 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} allocated sequence 0 03:04:29,449 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@785bcc58 03:04:29,449 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915855480847} 03:04:29,449 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}} 03:04:29,449 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 456 bytes, total slices to send: 1 03:04:29,449 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,449 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 changed state from TX_OPEN to IDLE 03:04:29,449 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@594a0db2 readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 03:04:29,449 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} allocated sequence 1 03:04:29,449 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@748a19c4 03:04:29,449 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915855642081} 03:04:29,449 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1c, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}} 03:04:29,450 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,450 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 03:04:29,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 17, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 03:04:29,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@4b5aa498 03:04:29,450 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 ready 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 to ready 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 is already in state READY (READY) 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct canCommit 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 validated 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct preCommit 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 candidate ready 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - candidate rootPath: / 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - messages: [] 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: preCommit - successfulFromPrevious: [] 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 prepared 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct commit 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 18, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 15, leaderLastIndex: 18, leaderSnapShotIndex: 13 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 15 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}], SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 submitted to persistence 03:04:29,451 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,452 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 03:04:29,452 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 03:04:29,452 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY}, enqueuedTicks=915855480847, sessionId=0, txSequence=27} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 03:04:29,452 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0} 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}], SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 14, entries to be added from: 0 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=18, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,452 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 18, nextIndex: 19 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 18, lastAppliedIndex: 14 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 15 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@7da40320 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 16 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@3bc35f87 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 17 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2114/0x00000008016ceeb8@637c59ba 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 18 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: commit - successfulFromPrevious: [] 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from COMMIT_PENDING to COMMITTED 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 03:04:29,453 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,453 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 03:04:29,453 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915851917988, sessionId=0, txSequence=23} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 03:04:29,453 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE}: purge completed 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 03:04:29,453 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} 03:04:29,453 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 63.89 μs 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 19, leaderLastIndex: 18, leaderSnapShotIndex: 17 03:04:29,453 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915852217795, sessionId=0, txSequence=24} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE}: purge completed 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915855226680, sessionId=0, txSequence=26} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN}: initiating purge 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} allocated sequence 2 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@5f058b8b 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915860049960} 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 18 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 18, context.getLastApplied(): 14, lastIndex(): 18 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 15 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=SIMPLE}, enqueuedTicks=915855642081, sessionId=0, txSequence=28} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} directCommit completed 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED}: initiating purge 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} allocated sequence 2 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@15e81356 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 03:04:29,454 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915860205404} 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 16 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 17 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 18 03:04:29,454 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 03:04:29,454 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 0 03:04:29,454 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@72c6c82a 03:04:29,454 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915860398970} 03:04:29,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,455 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 03:04:29,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 03:04:29,455 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 03:04:29,455 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 67.51 μs 03:04:29,455 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 03:04:29,455 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,455 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,455 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 19, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,455 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 19 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,455 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}]]] 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 20, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6f5558d6 03:04:29,456 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,456 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message size: 471 bytes, total slices to send: 1 03:04:29,456 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message does not need to be sliced - sending original message 03:04:29,456 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 03:04:29,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}]]] 03:04:29,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:04:29,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 18, entries to be added from: 0 03:04:29,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 03:04:29,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 03:04:29,456 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=19, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,457 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 19, nextIndex: 20 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 19, lastAppliedIndex: 18 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 19 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@40db60bd 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 03:04:29,457 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 18 for term 4 03:04:29,457 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=915860398970, sessionId=0, txSequence=31} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 03:04:29,457 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0} 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 20, leaderLastIndex: 20, leaderSnapShotIndex: 18 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 20 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=19, prevLogTerm=4, leaderCommit=19, replicatedToAllIndex=18, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}]]] 03:04:29,457 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} 03:04:29,457 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 1 03:04:29,457 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=19 03:04:29,457 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@1c74c3cf 03:04:29,457 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915863429155} 03:04:29,457 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=20, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}} 03:04:29,457 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,457 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,458 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=19, prevLogTerm=4, leaderCommit=19, replicatedToAllIndex=18, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}]]] 03:04:29,458 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915860049960, sessionId=0, txSequence=29} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:04:29,458 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE}: purge completed 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 19, entries to be added from: 0 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 03:04:29,458 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 19 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=20, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 19, context.getLastApplied(): 18, lastIndex(): 20 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 19 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 03:04:29,458 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 18 for term 4 03:04:29,459 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 03:04:29,459 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=19 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 20, nextIndex: 21 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 20, lastAppliedIndex: 19 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 20 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@6f30652b 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 19 for term 4 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 21, leaderLastIndex: 20, leaderSnapShotIndex: 19 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=20 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 21, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 03:04:29,462 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,463 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 03:04:29,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 20 03:04:29,463 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915860205404, sessionId=0, txSequence=30} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 03:04:29,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 20, context.getLastApplied(): 19, lastIndex(): 20 03:04:29,463 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE}: purge completed 03:04:29,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 20 03:04:29,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 03:04:29,463 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} 03:04:29,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 03:04:29,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 03:04:29,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 19 for term 4 03:04:29,463 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=20 03:04:29,464 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 21 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,464 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=20, prevLogTerm=4, leaderCommit=20, replicatedToAllIndex=19, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 03:04:29,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=20, prevLogTerm=4, leaderCommit=20, replicatedToAllIndex=19, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 03:04:29,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:04:29,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 20, entries to be added from: 0 03:04:29,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 03:04:29,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 03:04:29,464 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=21, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 21, nextIndex: 22 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 21, lastAppliedIndex: 20 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 21 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2114/0x00000008016ceeb8@4358d252 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 22, leaderLastIndex: 21, leaderSnapShotIndex: 20 03:04:29,465 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 03:04:29,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 21 03:04:29,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=915863429155, sessionId=0, txSequence=32} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 21, context.getLastApplied(): 20, lastIndex(): 21 03:04:29,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1} 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 21 03:04:29,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN}: initiating purge 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 03:04:29,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} allocated sequence 2 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 03:04:29,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@329f4960 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 03:04:29,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:29,465 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 03:04:29,465 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915871398913} 03:04:29,466 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 03:04:29,466 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,466 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,466 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 22, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:29,467 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,595 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 22 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,595 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 03:04:29,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 03:04:29,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 03:04:29,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 21, entries to be added from: 0 03:04:29,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 03:04:29,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 03:04:29,596 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=22, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:29,597 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 22, nextIndex: 23 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 22, lastAppliedIndex: 21 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 22 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@2688ca55 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 03:04:29,597 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 03:04:29,598 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 22 03:04:29,598 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 03:04:29,598 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 22, context.getLastApplied(): 21, lastIndex(): 22 03:04:29,598 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 22 03:04:29,598 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 03:04:29,598 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-490769927]}, enqueuedTicks=915871398913, sessionId=0, txSequence=33} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 03:04:29,598 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 03:04:29,598 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE}: purge completed 03:04:29,598 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} 03:04:29,598 PM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 03:04:29,598 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 03:04:29,598 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 03:04:29,958 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] 03:04:29,958 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 03:04:29,958 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722] 03:04:29,958 PM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722] 03:04:29,962 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 03:04:29,962 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 0 03:04:29,962 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@3a75f9d1 03:04:29,962 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:29,962 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=916367926646} 03:04:29,963 PM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 03:04:29,963 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,963 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6f5558d6 03:04:29,963 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 03:04:29,964 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message size: 460 bytes, total slices to send: 1 03:04:29,964 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-398: Message does not need to be sliced - sending original message 03:04:29,965 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 03:04:29,965 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-490769927], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=916367926646, sessionId=0, txSequence=34} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 03:04:29,965 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0} 03:04:29,965 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@44c2ba86 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} 03:04:29,965 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 1 03:04:29,965 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@4c38195 03:04:29,965 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:29,965 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}, enqueuedTicks=916370872299} 03:04:29,965 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-490769927], modifications=0, protocol=ABORT}} 03:04:29,965 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message size: 297 bytes, total slices to send: 1 03:04:29,965 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-399: Message does not need to be sliced - sending original message 03:04:29,965 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:04:29,965 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:29,965 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 03:04:29,965 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 03:04:29,965 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 03:04:29,965 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e2e320a 03:04:29,965 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 03:04:29,966 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e2e320a 03:04:29,966 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 03:04:29,966 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 03:04:29,966 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 03:04:29,966 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-490769927], modifications=0, protocol=ABORT}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true, paused: true 03:04:29,966 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:29,966 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:29,966 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: request Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-490769927], modifications=0, protocol=ABORT}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:29,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession] 03:04:29,967 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=13] 03:04:29,967 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@37abafdf, splicing queue 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@ad96251 queue spliced 1 messages 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: refreshing backend for shard 1 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:29,967 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#705386147] 03:04:29,967 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:29,967 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:29,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, matchIndex: 22, lastIndex: 22, isVoting: true 03:04:29,967 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Follower's log matches - sending ElectionTimeout 03:04:29,968 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 03:04:29,968 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leader transfer complete 03:04:30,195 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:04:30,385 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 03:04:30,385 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-1542675722]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:30,385 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:30,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-401: Closing 03:04:30,385 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 03:04:30,386 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 03:04:30,386 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 03:04:30,386 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 03:04:30,386 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 03:04:30,386 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 03:04:30,386 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#620175241] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#620175241] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:30,386 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:30,386 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 03:04:30,386 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:30,386 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1493607083] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:30,387 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#620175241] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#620175241] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:30,387 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:30,387 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:30,392 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:30,392 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:30,392 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:30,392 PM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:30,393 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:30,406 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 03:04:30,407 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 03:04:30,407 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 03:04:30,407 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 03:04:30,407 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 03:04:30,408 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:30,408 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:31,406 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2835405734656160986] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:31,426 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1801327992407967497] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:31,429 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:31,429 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:31,431 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:31.433] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:31,449 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:31,449 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:31,451 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:31.452] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:31,474 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:04:31,481 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3164395153376130234] 03:04:31,481 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:31,482 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:31,482 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:31,483 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:31,483 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:31,493 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:31,493 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:31,493 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:31,497 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:31,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-751965826465481601] 03:04:31,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:31,511 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:31,511 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:31,511 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:31,511 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:31,515 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@61e70212 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 03:04:31,515 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:04:31,515 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@147c84a3 03:04:31,515 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 03:04:31,515 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} allocated sequence 2 03:04:31,515 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1186768436], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2107/0x00000008016ccb68@6d5ee0d6 03:04:31,515 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:31,515 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#-1483264868], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:31,526 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:31,531 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7896623343980753021] 03:04:31,532 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:31,533 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:31,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:31,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:31,533 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:31,542 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:31,544 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:31,545 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 03:04:31,549 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:31,549 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:31,563 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:31,563 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,564 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 03:04:31,564 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:04:31,564 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6869d422 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3f9aad86 03:04:31,564 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@104725d5 03:04:31,564 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:31,564 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,564 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709823871564, None)) 03:04:31,565 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823871563,0,0)) 03:04:31,565 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1e98717 03:04:31,565 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3f9aad86 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5c9a3857 03:04:31,565 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 03:04:31,566 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:31,567 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:31,569 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 03:04:31,570 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:31,570 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,570 PM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,570 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,570 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739] 03:04:31,570 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 03:04:31,571 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:31,571 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-365431412 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,571 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-140431820] 03:04:31,572 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-365431412] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:31,572 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#-140431820] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:31,572 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 03:04:31,575 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:31,575 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:31,575 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:04:31,575 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:04:31,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 03:04:31,576 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 03:04:31,578 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 03:04:31,578 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 03:04:31,581 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 03:04:31,585 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:31,585 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1454113354] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:31,595 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:31,595 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 03:04:31,596 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:04:31,596 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,596 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@49fe99b4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7849486b 03:04:31,596 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3117ab61 03:04:31,596 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709823871596, None)) 03:04:31,596 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823871595,0,0)) 03:04:31,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5818db3d 03:04:31,597 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7849486b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@14ebc4eb 03:04:31,597 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 03:04:31,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:31,597 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,598 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 03:04:31,598 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#1440351408] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:31,598 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-811339159 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,598 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-477399453] 03:04:31,598 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-811339159] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:31,599 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-477399453] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:31,599 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 03:04:31,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:31,600 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:31,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:04:31,600 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:04:31,600 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 03:04:31,600 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 03:04:31,600 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 03:04:31,600 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 03:04:31,600 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 03:04:31,603 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:31,603 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:31,606 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:31,606 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:31,606 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,606 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,606 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,607 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:31,647 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] 03:04:31,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 03:04:31,648 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739] 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:04:31,810 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 03:04:31,810 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 03:04:32,40 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 03:04:32,40 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,40 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 03:04:32,145 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:32,279 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 03:04:32,279 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,280 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 03:04:32,501 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:32,501 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:32,501 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:32,501 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,501 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,501 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:32,501 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,501 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,501 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,501 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,505 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:32,505 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,505 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:32,505 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,505 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,505 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,553 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:32,600 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 03:04:32,600 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,600 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 03:04:32,600 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,600 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,602 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:32,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 03:04:32,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:32,602 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:32,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 03:04:32,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:32,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:32,603 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:32,603 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 03:04:32,604 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 03:04:32,604 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 03:04:32,605 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,606 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 03:04:32,606 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,606 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 03:04:32,606 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:32,606 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:32,606 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 03:04:32,606 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 03:04:32,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 03:04:32,606 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:32,640 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2e9a88de: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:04:32,641 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 03:04:32,641 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:32,641 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:32,641 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] 03:04:32,641 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:32,641 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4ea6382a] 03:04:32,641 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4ea6382a] 03:04:32,641 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} shard 1 03:04:32,641 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:04:32,641 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:32,642 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2e9a88de readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 03:04:32,642 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@44de83d2, attempting to connect 03:04:32,642 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@cf879bb 03:04:32,642 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} is not yet known 03:04:32,642 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=14291, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:04:32,642 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1ea8e3d7 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7ffb0e02 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7ffb0e02 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], modifications=1, protocol=SIMPLE} 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} allocated sequence 0 03:04:32,643 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@cf879bb 03:04:32,644 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:32,644 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@bed1157 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], coordinated=false}, enqueuedTicks=919047888709} 03:04:32,644 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2e9a88de finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:32,644 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 03:04:32,644 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5282fda6 03:04:32,644 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@752e72ea superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@166f05bc, splicing queue 03:04:32,644 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@752e72ea queue spliced 0 messages 03:04:32,644 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.050 ms 03:04:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct canCommit 03:04:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 validated 03:04:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct preCommit 03:04:32,644 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 candidate ready 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - messages: [] 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 prepared 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct commit 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:32,645 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,646 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 03:04:32,646 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 submitted to persistence 03:04:32,646 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:32,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 03:04:32,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 03:04:32,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:32,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 03:04:32,647 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:32,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 03:04:32,647 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:32,648 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:32,648 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@20c73091}, purged=MutableUnsignedLongSet{size=0}} 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#545862178] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:32,648 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], coordinated=false} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], coordinated=false}, enqueuedTicks=919047888709, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#545862178] 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@756bc98f 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#545862178] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED}: initiating purge 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} allocated sequence 1 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@30c16549 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@bed1157 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]}, enqueuedTicks=919054779300} 03:04:32,649 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 91.16 μs 03:04:32,649 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,649 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 03:04:32,649 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 03:04:32,649 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:32,649 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 03:04:32,649 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:32,649 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 03:04:32,649 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 03:04:32,649 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:32,649 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 03:04:32,649 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,650 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f82a14e}, purged=MutableUnsignedLongSet{size=0}} 03:04:32,650 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:32,650 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,650 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#113430586] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 03:04:32,650 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 03:04:32,650 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#113430586] 03:04:32,650 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:04:32,650 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 57.75 μs 03:04:32,650 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#113430586] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,651 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,651 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 03:04:32,651 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:32,653 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 03:04:32,656 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1276907430] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,657 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:32,657 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 03:04:32,657 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 03:04:32,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 03:04:32,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:32,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 03:04:32,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 03:04:32,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 03:04:32,658 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:32,659 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: replication of member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@fe7e0e 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: finished purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:32,659 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]}, enqueuedTicks=919054779300, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE}: purge completed 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7ffb0e02 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1ea8e3d7 purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 03:04:32,660 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2f3b86c7 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4465786a 03:04:32,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 03:04:32,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:04:32,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 03:04:32,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 03:04:32,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 03:04:32,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:32,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:32,660 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 03:04:32,661 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1ede6d76 03:04:32,661 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1709823872660, None)) 03:04:32,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:32,661 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823872659,0,0)) 03:04:32,661 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,661 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4465786a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@56824953 03:04:32,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:32,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:32,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:32,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1de643a8 03:04:32,662 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,662 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1276907430] 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#932173294] 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#932173294] 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,662 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#932173294] 03:04:32,662 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1454113354] 03:04:32,662 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2e9a88de: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:04:32,662 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 03:04:32,663 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#932173294] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,663 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:04:32,663 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#2117761696 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,663 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-1750988454] 03:04:32,663 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#2117761696] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,663 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] 03:04:32,663 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 03:04:32,663 PM [pool-5-thread-1] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2e9a88de readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 03:04:32,663 PM [pool-5-thread-1] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 03:04:32,663 PM [pool-5-thread-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@7ab13f71 03:04:32,663 PM [pool-5-thread-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:32,663 PM [pool-5-thread-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@bed1157 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], coordinated=false}, enqueuedTicks=919069598691} 03:04:32,666 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-1750988454] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:32,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:32,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct canCommit 03:04:32,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:32,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 validated 03:04:32,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:32,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct preCommit 03:04:32,666 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 candidate ready 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - candidate rootPath: / 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - messages: [] 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 prepared 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct commit 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 72 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 03:04:32,667 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 submitted to persistence 03:04:32,668 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 03:04:32,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:32,668 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:32,668 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 03:04:32,668 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 03:04:32,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:04:32,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:04:32,668 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 03:04:32,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 03:04:32,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 03:04:32,668 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,669 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 03:04:32,671 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 03:04:32,671 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:32,671 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,671 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 03:04:32,672 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,672 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:32,672 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 03:04:32,672 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 03:04:32,672 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 03:04:32,672 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:32,672 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:32,672 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 03:04:32,672 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 03:04:32,765 PM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739] 03:04:32,766 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739] 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-406: Closing 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 03:04:32,768 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:32,768 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 03:04:32,768 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:32,768 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@bed1157 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1e9dda5, splicing queue 03:04:32,769 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:32,769 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@bed1157 queue spliced 1 messages 03:04:32,769 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:32,769 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 03:04:32,769 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:32,769 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:32,769 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:32,769 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] 03:04:32,769 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:32,769 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 03:04:33,20 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 03:04:33,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 03:04:33,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 03:04:33,21 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 03:04:33,21 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 03:04:33,21 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 03:04:33,21 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:33,21 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 03:04:33,22 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 03:04:33,22 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 03:04:33,22 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 03:04:33,22 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 03:04:33,22 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,22 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 03:04:33,23 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:33,23 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 03:04:33,23 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 03:04:33,23 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 03:04:33,23 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 03:04:33,23 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:33,23 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,23 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 03:04:33,24 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=13] 03:04:33,24 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 03:04:33,24 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 03:04:33,24 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:33,25 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 03:04:33,25 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 03:04:33,25 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 03:04:33,25 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:33,25 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:33,25 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:33,25 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 03:04:33,25 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:33,25 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:33,26 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:33,26 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 03:04:33,26 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:33,26 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 03:04:33,26 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:33,27 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 03:04:33,27 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 03:04:33,28 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 03:04:33,28 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@48614ff5}, purged=MutableUnsignedLongSet{size=0}} 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-544428394] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 03:04:33,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-544428394] 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:33,29 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 03:04:33,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:04:33,29 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-544428394] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 03:04:33,29 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 114.7 μs 03:04:33,29 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:33,30 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 03:04:33,30 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:33,30 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 03:04:33,30 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 03:04:33,30 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 03:04:33,30 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:33,30 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, size=72} 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: commit - successfulFromPrevious: [] 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 03:04:33,30 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} not found in inflight queue, checking pending queue 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], coordinated=false} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]] - lastApplied: 3, lastIndex: 3 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 1 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470], coordinated=false}, enqueuedTicks=919069598691, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} directCommit completed 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED}: initiating purge 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 1 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-409: Closing 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@424fc4f 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:33,31 PM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 03:04:33,31 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 46.35 μs 03:04:33,31 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 03:04:33,31 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is ready - notifying 1 OnShardInitialized callbacks 03:04:33,31 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4ea6382a] 03:04:33,31 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 03:04:33,32 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4ea6382a] 03:04:33,32 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 03:04:33,32 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 03:04:33,32 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@a2fae7f, attempting to connect 03:04:33,32 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 03:04:33,32 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 03:04:33,32 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:04:33,32 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:33,33 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:04:33,33 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 1 for term 6 03:04:33,33 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 47.64 μs 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] is present: existingEntryTerm: 4 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] is present: existingEntryTerm: 6 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 2 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7ffb0e02 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@36fee899 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} previous state DONE 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@36fee899 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 03:04:33,33 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN}: initiating purge 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 0 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@58d2491f 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 03:04:33,33 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@73ded43b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]}, enqueuedTicks=919437053351} 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 18.86 μs 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2e9a88de finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@488e1c24 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@9f59f8c superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@43a5b51c, splicing queue 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@9f59f8c queue spliced 0 messages 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.014 ms 03:04:33,34 PM [cluster-test-shard-dispatcher-11] [WARN] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 not tracked in MutableUnsignedLongSet{span=[0..0], size=1}, but not present in active transactions 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1926669470]}, enqueuedTicks=919437053351, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE}: purge completed 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@36fee899 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 03:04:33,34 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@7ffb0e02 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 03:04:33,69 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:04:33,70 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,70 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 03:04:33,70 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 03:04:33,70 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 03:04:33,70 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 03:04:33,71 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 03:04:33,71 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:33,71 PM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:04:33,71 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:04:33,71 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 03:04:33,71 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:04:33,72 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,72 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 03:04:33,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 03:04:33,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 03:04:33,72 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1276907430] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1276907430] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:33,72 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#993810530] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#993810530] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:33,72 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 03:04:33,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@54a33849 03:04:33,72 PM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:33,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@54a33849 03:04:33,72 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:33,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 03:04:33,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:04:33,72 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:04:33,73 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@73ded43b superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@5526ea03, splicing queue 03:04:33,73 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@73ded43b queue spliced 0 messages 03:04:33,73 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:33,73 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 03:04:33,73 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:33,73 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:33,73 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:33,73 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] 03:04:33,73 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:33,73 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 03:04:33,74 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 03:04:33,74 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop] 03:04:33,74 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=13] 03:04:33,74 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:33,74 PM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:33,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, matchIndex: 3, lastIndex: 3, isVoting: true 03:04:33,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Follower's log matches - sending ElectionTimeout 03:04:33,78 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:04:33,79 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:04:33,79 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leader transfer complete 03:04:33,490 PM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 03:04:33,490 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received ShardNotInitializedTimeout message for shard cars 03:04:33,490 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-471532739]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:33,490 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Returning NoShardLeaderException for shard cars 03:04:33,491 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:33,491 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:33,491 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:33,491 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,495 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-410: Closing 03:04:33,492 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:33,495 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 03:04:33,495 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:33,495 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:04:33,496 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 03:04:33,496 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:04:33,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 03:04:33,496 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:33,496 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:04:33,496 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:33,496 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:33,496 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1046001764] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:33,497 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:33,497 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:33,501 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:33,501 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:33,501 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:33,501 PM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:33,503 PM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:33,511 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 03:04:33,512 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 03:04:33,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 03:04:33,512 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 03:04:33,512 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 03:04:33,514 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:33,514 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:34,513 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3164395153376130234] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:34,527 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:34,528 PM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:34,529 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:34.530] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:34,533 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-751965826465481601] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:34,538 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:34,538 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:34,539 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:34.540] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:34,564 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:34,571 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1990027147993580460] 03:04:34,571 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:34,573 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:34,573 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:34,574 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:34,574 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:34,586 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:34,586 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:34,586 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:34,587 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:04:34,592 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2192714618066261624] 03:04:34,592 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:34,593 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:34,593 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:34,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:34,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:34,609 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:04:34,618 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8280057387342155631] 03:04:34,618 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:34,618 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:34,618 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:34,623 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:34,625 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:34,626 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 03:04:34,629 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:34,629 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:34,636 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 03:04:34,640 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-784507526] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:34,640 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 03:04:34,640 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:04:34,640 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@29d43f5c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6cd6a402 03:04:34,640 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 03:04:34,640 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,641 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:34,641 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,642 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:34,642 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2d1a0080 03:04:34,642 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1709823874640, None)) 03:04:34,642 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823874639,0,0)) 03:04:34,642 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6cd6a402 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@55561351 03:04:34,643 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 03:04:34,643 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:34,645 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:34,646 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:34,647 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 03:04:34,648 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3593f857 03:04:34,648 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:34,648 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,649 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,649 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,649 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079] 03:04:34,649 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 03:04:34,649 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:34,650 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#1629152254] 03:04:34,650 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-877249334 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,650 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#-877249334] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:34,650 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#1629152254] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:34,650 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 03:04:34,650 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:34,650 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:34,650 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 03:04:34,650 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 03:04:34,650 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:34,650 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 03:04:34,650 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 03:04:34,651 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 03:04:34,651 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 03:04:34,652 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-496518647] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:34,652 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 03:04:34,655 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:34,656 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 03:04:34,656 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 03:04:34,656 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:04:34,656 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 03:04:34,656 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@683b622c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@66fc915b 03:04:34,656 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@62c3095c 03:04:34,657 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:34,657 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,657 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1709823874657, None)) 03:04:34,657 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823874656,0,0)) 03:04:34,657 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@186766b1 03:04:34,657 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@66fc915b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@278521d2 03:04:34,657 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 03:04:34,658 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:34,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:34,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 03:04:34,658 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:04:34,658 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-2-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-728483121] 03:04:34,658 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 03:04:34,658 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-728483121] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:34,659 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-2102375158 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 03:04:34,659 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 03:04:34,659 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#-2102375158] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:34,659 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#1296310285] 03:04:34,659 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#1296310285] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:34,660 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:34,660 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:34,660 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 03:04:34,660 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 03:04:34,660 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:34,660 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 03:04:34,660 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 03:04:34,661 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,661 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 03:04:34,661 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 03:04:34,708 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-784507526] 03:04:34,708 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 03:04:34,708 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079] 03:04:34,897 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 03:04:34,898 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 03:04:34,898 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 03:04:35,157 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 03:04:35,157 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:04:35,157 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 03:04:35,214 PM [cluster-test-shard-dispatcher-20] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:35,437 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 03:04:35,437 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:04:35,437 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 03:04:35,588 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:35,588 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:35,588 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:35,588 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 03:04:35,588 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:35,588 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:35,588 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,588 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:35,588 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,593 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:35,593 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 03:04:35,593 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,593 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:35,593 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,593 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,627 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:35,757 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 03:04:35,757 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:04:35,757 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 4 03:04:35,757 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,759 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:35,759 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 03:04:35,759 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:35,759 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:35,759 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=null 03:04:35,759 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:35,759 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 03:04:35,759 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:35,761 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:35,761 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:35,761 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 03:04:35,761 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:35,761 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:35,761 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:04:35,761 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 03:04:35,761 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 03:04:35,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:35,761 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:35,761 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 03:04:35,761 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 03:04:35,761 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 03:04:35,762 PM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 03:04:35,762 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 03:04:35,762 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 03:04:35,762 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=13] 03:04:35,762 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:35,762 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:35,762 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 03:04:35,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 03:04:35,763 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:35,766 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 03:04:35,766 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 changed state from IDLE to TX_OPEN 03:04:35,766 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 03:04:35,766 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:35,766 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:35,766 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-496518647] 03:04:35,766 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:35,766 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} shard 1 03:04:35,766 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 03:04:35,766 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:35,767 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:04:35,767 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=13] 03:04:35,767 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:35,767 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@5cc28208 03:04:35,767 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 changed state from TX_OPEN to IDLE 03:04:35,767 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:04:35,767 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 03:04:35,767 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 03:04:35,767 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@69fec435 03:04:35,769 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@76ba0718, attempting to connect 03:04:35,770 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} is not yet known 03:04:35,770 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=9880, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:04:35,771 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@23321027 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5dc43c88 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state SEALED 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} previous state OPEN 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902]}, enqueuedTicks=922172596692} 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5dc43c88 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 03:04:35,771 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2043/0x00000008016a3198@58087c2e 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=1, protocol=READY}, enqueuedTicks=922172817297} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=1, protocol=READY}} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message size: 389 bytes, total slices to send: 1 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message does not need to be sliced - sending original message 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5dc43c88 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2089/0x00000008016bf9a8@b6d27cf 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=922173070214} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@61b2f50f 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@f71899b superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@eec7e20, splicing queue 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@f71899b queue spliced 0 messages 03:04:35,772 PM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.503 ms 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@269fa47d 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 ready 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 to ready 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,774 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}] 03:04:35,775 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 03:04:35,775 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: Message size: 331 bytes, total slices to send: 1 03:04:35,775 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: Message does not need to be sliced - sending original message 03:04:35,775 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,775 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 03:04:35,776 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=1, protocol=READY}, enqueuedTicks=922172817297, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 03:04:35,776 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0} 03:04:35,776 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 03:04:35,776 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=922173070214, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 03:04:35,776 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0} 03:04:35,776 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 changed state from IDLE to TX_OPEN 03:04:35,776 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:04:35,776 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 03:04:35,776 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@4df2f084 03:04:35,776 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=922182159661} 03:04:35,778 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,778 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,778 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}] 03:04:35,778 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 03:04:35,778 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: Message size: 331 bytes, total slices to send: 1 03:04:35,778 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: Message does not need to be sliced - sending original message 03:04:35,779 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 03:04:35,779 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=922182159661, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 03:04:35,779 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0} 03:04:35,779 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 03:04:35,779 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 2 03:04:35,779 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@782024ef 03:04:35,779 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2}, enqueuedTicks=922185323919} 03:04:35,779 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2}} 03:04:35,779 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message size: 459 bytes, total slices to send: 1 03:04:35,779 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message does not need to be sliced - sending original message 03:04:35,780 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2090/0x00000008016c4000@2314d66b 03:04:35,780 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=922185756101} 03:04:35,782 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 03:04:35,782 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2}, enqueuedTicks=922185323919, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 03:04:35,782 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1} 03:04:35,782 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 03:04:35,782 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=922185756101, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 03:04:35,782 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true} 03:04:35,782 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 3 03:04:35,782 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@8a2f7af 03:04:35,782 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=922188133580} 03:04:35,783 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 03:04:35,783 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: Message size: 369 bytes, total slices to send: 1 03:04:35,783 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: Message does not need to be sliced - sending original message 03:04:35,784 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 03:04:35,784 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=922188133580, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 03:04:35,784 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3} 03:04:35,784 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 4 03:04:35,784 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 03:04:35,784 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@39121780 03:04:35,784 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE}, enqueuedTicks=922190268853} 03:04:35,784 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE}} 03:04:35,784 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message size: 285 bytes, total slices to send: 1 03:04:35,784 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message does not need to be sliced - sending original message 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 is already in state READY (READY) 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct canCommit 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 validated 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct preCommit 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 candidate ready 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - messages: [] 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 prepared 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct commit 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 03:04:35,787 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 03:04:35,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 03:04:35,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:35,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:35,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}]]] 03:04:35,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 submitted to persistence 03:04:35,788 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 03:04:35,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}]]] 03:04:35,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 03:04:35,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:35,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 03:04:35,788 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,789 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=174} 03:04:35,789 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 03:04:35,789 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 03:04:35,789 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:35,789 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,789 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 03:04:35,789 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 1, nextIndex: 2 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2048/0x00000008016a84c0@193a0632 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3be14f49}, purged=MutableUnsignedLongSet{size=0}} 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 03:04:35,789 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:04:35,790 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 03:04:35,790 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#28760555] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 03:04:35,790 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:04:35,790 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:35,790 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 03:04:35,791 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#28760555] 03:04:35,791 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 03:04:35,791 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#28760555] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:35,791 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 163.7 μs 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 1 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@e7c41c9}, purged=MutableUnsignedLongSet{size=0}} 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 03:04:35,791 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 03:04:35,792 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 03:04:35,792 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,792 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,792 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1951388590] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 03:04:35,792 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:04:35,792 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 03:04:35,792 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1951388590] 03:04:35,792 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 03:04:35,792 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1951388590] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:35,792 PM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 69.79 μs 03:04:35,792 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 03:04:35,792 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902]}, enqueuedTicks=922172596692, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 03:04:35,792 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 03:04:35,793 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 03:04:35,793 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE}, enqueuedTicks=922190268853, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 03:04:35,793 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} directCommit completed 03:04:35,793 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED}: initiating purge 03:04:35,793 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 03:04:35,793 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@5f4c9670 03:04:35,793 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@2dae3d18 03:04:35,793 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:35,793 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902]}, enqueuedTicks=922198742454} 03:04:35,793 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2, protocol=READY}, enqueuedTicks=922198828996} 03:04:35,793 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2, protocol=READY}} 03:04:35,793 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message size: 473 bytes, total slices to send: 1 03:04:35,793 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message does not need to be sliced - sending original message 03:04:35,793 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 changed state from TX_OPEN to IDLE 03:04:35,793 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:04:35,793 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} allocated sequence 5 03:04:35,793 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@69c31b6c 03:04:35,793 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE}, enqueuedTicks=922199321969} 03:04:35,793 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE}} 03:04:35,793 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message size: 286 bytes, total slices to send: 1 03:04:35,793 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message does not need to be sliced - sending original message 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 ready 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 to ready 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 is already in state READY (READY) 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct canCommit 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 validated 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct preCommit 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 candidate ready 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - candidate rootPath: / 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - messages: [] 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: preCommit - successfulFromPrevious: [] 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 prepared 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct commit 03:04:35,794 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 230 03:04:35,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 03:04:35,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 03:04:35,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:35,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 03:04:35,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}]]] 03:04:35,795 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 submitted to persistence 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}]]] 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=230} 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:35,796 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 03:04:35,796 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2, protocol=READY}, enqueuedTicks=922198828996, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 03:04:35,796 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4} 03:04:35,796 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 03:04:35,796 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 03:04:35,796 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 0 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@9e0e8f9 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: commit - successfulFromPrevious: [] 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 03:04:35,797 PM [cluster-test-shard-dispatcher-12] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 03:04:35,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 03:04:35,798 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 52.66 μs 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902]} with Envelope{sessionId=0, txSequence=8, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902]}, enqueuedTicks=922198742454, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5dc43c88 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=SIMPLE}, enqueuedTicks=922199321969, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} directCommit completed 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED}: initiating purge 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 6 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#691252902]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@9d809eb 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:35,798 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 03:04:35,798 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#691252902]}, enqueuedTicks=922204447716} 03:04:35,798 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@23321027 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 03:04:35,798 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 1 03:04:35,798 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@55fec9e3 03:04:35,798 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 03:04:35,798 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} allocated sequence 1 03:04:35,798 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2107/0x00000008016ccb68@4e39e85a 03:04:35,799 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=ABORT}, enqueuedTicks=922204679693} 03:04:35,799 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=c, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=ABORT}} 03:04:35,799 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message size: 286 bytes, total slices to send: 1 03:04:35,799 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-413: Message does not need to be sliced - sending original message 03:04:35,799 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 03:04:35,799 PM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 03:04:35,799 PM [main] [WARN] ClientBackedTransactionChain - Aborted unclosed transactions [member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0] java.lang.Throwable: at at org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain.close(ClientBackedTransactionChain.java:76) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testTransactionChainWithSingleShard(DistributedDataStoreRemotingIntegrationTest.java:581) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 03:04:35,799 PM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5dc43c88 invoking destroy 03:04:35,799 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902]}, enqueuedTicks=922205438222} 03:04:35,799 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0 changed state from IDLE to CLOSED 03:04:35,799 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@301be147: creating a new snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 03:04:35,799 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0} shard 1 03:04:35,799 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 03:04:35,799 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:35,800 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@160149cc 03:04:35,800 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=922205694468} 03:04:35,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 03:04:35,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 03:04:35,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:35,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: aborting transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 03:04:35,801 PM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 03:04:35,801 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 03:04:35,801 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: Message size: 369 bytes, total slices to send: 1 03:04:35,801 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-412: Message does not need to be sliced - sending original message 03:04:35,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 03:04:35,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 03:04:35,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,803 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 03:04:35,803 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#691252902], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=922205694468, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 03:04:35,803 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0} 03:04:35,803 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:04:35,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 03:04:35,803 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@13fbc94a 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@13fbc94a 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 6, leaderSnapShotIndex: 2 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 03:04:35,803 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 03:04:35,804 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:04:35,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:35,804 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:35,804 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard] 03:04:35,804 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:04:35,804 PM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 3 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 6 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 6 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 03:04:35,804 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 03:04:35,805 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,805 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 03:04:35,805 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@421e0fdb, splicing queue 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 103.1 μs 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7840fb43 queue spliced 3 messages 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: refreshing backend for shard 1 03:04:35,805 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:35,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:35,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:35,806 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-496518647] 03:04:35,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 03:04:35,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 03:04:35,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 03:04:35,806 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 03:04:35,806 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:35,806 PM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 6, nextIndex: 7 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 3 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 4 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 3 to 4 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@1de8e21e 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 5 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 4 to 5 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2114/0x00000008016ceeb8@5014c970 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Aborted transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 6 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 5 to 6 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2073/0x00000008016bc240@6af138b5 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,806 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 5 for term 4 03:04:35,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 03:04:35,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionChainWithSingleShard, matchIndex: 6, lastIndex: 6, isVoting: true 03:04:35,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 03:04:35,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 03:04:35,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leader transfer complete 03:04:35,807 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=6 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} not found in inflight queue, checking pending queue 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#691252902]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#691252902]}, enqueuedTicks=922204447716, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6}} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5dc43c88 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} not found in inflight queue, checking pending queue 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=ABORT}, enqueuedTicks=922204679693, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1}} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=ABORT} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR}: initiating purge 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 2 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@646a2471 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED}: initiating purge 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#691252902]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@12c192e4 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} not found in inflight queue, checking pending queue 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902]} with Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902]}, enqueuedTicks=922205438222, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1}} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5dc43c88 destroy completed with LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1} 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@4d56f4c0: removed destroyed proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5dc43c88 03:04:35,808 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:35,808 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 6 03:04:35,808 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 3, lastIndex(): 6 03:04:35,808 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 4 03:04:35,808 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 3 to 4 03:04:35,808 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67} 03:04:35,808 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 03:04:35,808 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 5 03:04:35,808 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 4 to 5 03:04:35,809 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67} 03:04:35,809 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Aborted transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 03:04:35,809 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 6 03:04:35,809 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 5 to 6 03:04:35,809 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 6 data CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 03:04:35,809 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Closed history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 03:04:35,809 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 5 for term 4 03:04:35,809 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=6 03:04:36,218 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 03:04:36,218 PM [cluster-test-akka.actor.default-dispatcher-26] [WARN] ShardManager - shard-manager-testTransactionChainWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:36,218 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 03:04:36,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-415: Closing 03:04:36,218 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 03:04:36,218 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 03:04:36,218 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 03:04:36,219 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 03:04:36,219 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 03:04:36,219 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 03:04:36,219 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 03:04:36,219 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-784507526] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-784507526] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:36,219 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-1743374128] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1743374128] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:36,219 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-1743374128] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1743374128] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:36,219 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:36,219 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:36,225 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:36,225 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:36,225 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:04:36.226] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-437)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:36,240 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 03:04:36,240 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 03:04:36,240 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 03:04:36,240 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 03:04:36,241 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 03:04:36,241 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:36,241 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:37,237 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1990027147993580460] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:37,256 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:37,256 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:37,256 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2192714618066261624] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:37,258 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:37.259] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:37,276 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:37,276 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:37,278 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:37.280] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:37,291 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:37,296 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8578482303259281913] 03:04:37,297 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:37,299 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:37,299 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:37,302 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:37,302 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:37,310 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:37,310 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:37,310 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:37,314 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:37,318 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7196128891317250508] 03:04:37,318 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:37,321 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:37,321 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:37,330 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:37,330 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:37,336 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@47e8cf67 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} 03:04:37,336 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 03:04:37,336 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#691252902], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@7fa7853a 03:04:37,336 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-1281752079], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:37,336 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0} 03:04:37,341 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:37,355 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7189006096708583543] 03:04:37,355 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:37,355 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:37,355 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:37,355 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:37,355 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:37,377 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:37,377 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:37,379 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:37,385 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:37,389 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:37,389 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:37,397 PM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection closed with error: Connection reset 03:04:37,400 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:37,400 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:38,414 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:38.415] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:38,424 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:38,424 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [03/07/2024 15:04:38.427] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:38,452 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:38,460 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1702284269399300580] 03:04:38,460 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:38,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:38,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:38,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:38,462 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:38,474 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:38,474 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:38,474 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:38,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:38,497 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-7585292480282490916] 03:04:38,497 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:38,497 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:38,497 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:38,497 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:38,498 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:38,499 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:38,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-158679375157483061] 03:04:38,505 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:38,513 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:38,513 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:38,516 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:38,517 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:38,519 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 03:04:38,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:38,524 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:38,527 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 03:04:38,531 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:38,535 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 03:04:38,535 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,536 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:38,536 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,545 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 03:04:38,545 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:04:38,545 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@38ca326b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@494dab4c 03:04:38,545 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@9382f8d 03:04:38,545 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1709823878545, None)) 03:04:38,545 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823878544,0,0)) 03:04:38,546 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@70b95793 03:04:38,546 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@494dab4c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@58e5cd24 03:04:38,546 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 03:04:38,547 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:38,549 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:38,551 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 03:04:38,551 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:38,552 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:38,552 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,554 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:38,556 PM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,556 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,556 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920] 03:04:38,556 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 03:04:38,557 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 03:04:38,558 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1b7167ca to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2fa4e5e9 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4c9bcd4c 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1709823878559, None)) 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#913987462] 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1461242149 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1461242149] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#913987462] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823878558,0,0)) 03:04:38,559 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2fa4e5e9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@58c76832 03:04:38,559 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 03:04:38,560 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:38,560 PM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:38,560 PM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:38,560 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 03:04:38,560 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 03:04:38,560 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 03:04:38,560 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 03:04:38,560 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 03:04:38,561 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 03:04:38,561 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1949988542] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:38,562 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 03:04:38,571 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 03:04:38,571 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,571 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@c726be4 03:04:38,571 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:38,571 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,572 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 03:04:38,572 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#1770424078] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:38,573 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 03:04:38,573 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-259087812 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,573 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-259087812] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:38,574 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#690964192] 03:04:38,574 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#690964192] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:38,575 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:38,575 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:38,575 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 03:04:38,575 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:38,575 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 03:04:38,575 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 03:04:38,575 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 03:04:38,575 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 03:04:38,575 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 03:04:38,579 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:38,581 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:38,582 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:38,582 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,582 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,582 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,610 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] 03:04:38,610 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 03:04:38,610 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920] 03:04:38,816 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:38,816 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 03:04:38,816 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 03:04:38,816 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:38,816 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 03:04:38,816 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:38,816 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 03:04:38,816 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:38,817 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 03:04:38,817 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 03:04:38,817 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 03:04:39,115 PM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:39,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:04:39,135 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,135 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 03:04:39,365 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:04:39,365 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,365 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 03:04:39,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:39,477 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:39,478 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:39,478 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,478 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,478 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:39,478 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,478 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,478 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,520 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:39,520 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,520 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,520 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:39,520 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,520 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,527 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:39,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 03:04:39,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,665 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 03:04:39,665 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:39,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 03:04:39,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:39,671 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:39,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 03:04:39,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:39,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,671 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:39,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:39,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:39,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 03:04:39,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:39,672 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:39,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:04:39,672 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 03:04:39,672 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 03:04:39,673 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 03:04:39,673 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:39,673 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:39,673 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 03:04:39,673 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 03:04:39,673 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 03:04:39,673 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,673 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 03:04:39,674 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=13] 03:04:39,674 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:39,674 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:39,674 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 03:04:39,674 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 03:04:39,674 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:39,772 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1949988542] 03:04:39,773 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 03:04:39,773 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#1770424078] 03:04:39,773 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,773 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#1949988542] 03:04:39,773 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 03:04:39,773 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#1770424078] 03:04:39,785 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,785 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,785 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 03:04:39,788 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 03:04:39,788 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:04:39,788 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:04:39,788 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 03:04:39,788 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:39,788 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,788 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 03:04:39,788 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 03:04:39,788 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-175-0 of 3 operations, client version 13 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-175-0 ready 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-175-0 validated 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-175-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-175-0: success 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-175-0 candidate ready 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-175-0: canCommit - candidate rootPath: / 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-175-0: canCommit - messages: [] 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-175-0: preCommit - successfulFromPrevious: [] 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-175-0 prepared 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-175-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:04:39,789 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:04:39,790 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-175-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:39,790 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:39,790 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,790 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-175-0 submitted to persistence 03:04:39,790 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 03:04:39,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:04:39,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:04:39,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:39,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:04:39,792 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:04:39,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:04:39,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 03:04:39,792 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 03:04:39,793 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=5362873} is already disabled 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 03:04:39,793 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 03:04:39,794 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 03:04:39,794 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,795 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-175-0: commit - successfulFromPrevious: [] 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-175-0 changing state from COMMIT_PENDING to COMMITTED 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-175-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-215#-204001222] 03:04:39,795 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#314466854] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:39,795 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 03:04:39,795 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#314466854] 03:04:39,795 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 03:04:39,795 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 69.11 μs 03:04:39,795 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#314466854] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:39,796 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 03:04:39,796 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3957a2ad: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 03:04:39,796 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:04:39,796 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 03:04:39,796 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 03:04:39,796 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 03:04:39,796 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 03:04:39,796 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:39,796 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:39,796 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] 03:04:39,796 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-175-0 03:04:39,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:39,797 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 03:04:39,797 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6cdbd74c] 03:04:39,797 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 03:04:39,797 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:39,797 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1327611413], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@42bbd916 03:04:39,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#293263230] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 03:04:39,797 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:39,797 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#293263230] 03:04:39,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 03:04:39,797 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 03:04:39,797 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 03:04:39,797 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#293263230] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:39,797 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6cdbd74c] 03:04:39,797 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 82.80 μs 03:04:39,797 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@492252bb, attempting to connect 03:04:39,798 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} is not yet known 03:04:39,798 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, nanosAgo=4960, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:04:39,798 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@15fbb38 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@445d3577 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@445d3577 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1327611413], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3957a2ad finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@71cd8824 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7553530e superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@929b0e7, splicing queue 03:04:39,798 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7553530e queue spliced 0 messages 03:04:39,799 PM [cluster-test-akka.actor.default-dispatcher-14] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 531.0 μs 03:04:39,799 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 03:04:39,799 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,799 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,799 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 03:04:39,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-176-0 of 1 operations, client version 13 03:04:39,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,800 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-176-0 ready 03:04:39,802 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0 canCommit 03:04:39,802 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920)] resolved 03:04:39,802 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0: successfully resolved all cohort actors 03:04:39,802 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0 finishCanCommit 03:04:39,802 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-176-0, version=13] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920)] 03:04:39,802 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920)] 03:04:39,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-176-0 validated 03:04:39,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-176-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:39,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-176-0: success 03:04:39,802 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0: received CanCommitTransactionReply [canCommit=true, version=13] 03:04:39,802 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0: canCommit returning result: true 03:04:39,802 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0 commit 03:04:39,802 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0 finish commit 03:04:39,802 PM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-176-0, version=13] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920)] 03:04:39,802 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920)] 03:04:39,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,802 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-176-0 candidate ready 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-176-0: canCommit - candidate rootPath: / 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-176-0: canCommit - messages: [] 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-176-0: preCommit - successfulFromPrevious: [] 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-176-0 prepared 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-176-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-176-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 03:04:39,803 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-176-0 submitted to persistence 03:04:39,804 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:04:39,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 03:04:39,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 03:04:39,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:04:39,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 03:04:39,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:04:39,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 03:04:39,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:39,805 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 03:04:39,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 03:04:39,805 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-176-0: commit - successfulFromPrevious: [] 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-176-0 changing state from COMMIT_PENDING to COMMITTED 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-176-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23-904641920$a] 03:04:39,806 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-176-0: commit succeeded 03:04:39,806 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3957a2ad: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 03:04:39,806 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:04:39,806 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 03:04:39,806 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 03:04:39,806 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 68.31 μs 03:04:39,806 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:04:39,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:39,807 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 03:04:39,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 03:04:39,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 03:04:39,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c2add64 03:04:39,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7c2add64 03:04:39,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 03:04:39,807 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:04:39,807 PM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 03:04:39,808 PM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152958ae superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1beea47b, splicing queue 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@152958ae queue spliced 0 messages 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: refreshing backend for shard 1 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:39,808 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] 03:04:39,808 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:39,808 PM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 03:04:39,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 2 03:04:39,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 03:04:39,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 2 03:04:39,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 03:04:39,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 03:04:39,808 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-176-0 03:04:39,809 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 03:04:39,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 03:04:39,809 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 03:04:39,809 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 139.3 μs 03:04:39,809 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, matchIndex: 2, lastIndex: 2, isVoting: true 03:04:39,809 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Follower's log matches - sending ElectionTimeout 03:04:39,809 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:04:39,809 PM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leader transfer complete 03:04:39,810 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader] 03:04:39,810 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=13] 03:04:39,810 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:39,810 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:40,28 PM [cluster-test-shard-dispatcher-12] [WARN] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 217, lastApplied : 2, commitIndex : 2 03:04:40,225 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received ShardNotInitializedTimeout message for shard cars 03:04:40,225 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Returning NoShardLeaderException for shard cars 03:04:40,225 PM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 03:04:40,225 PM [cluster-test-akka.actor.default-dispatcher-14] [WARN] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:40,226 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:40,226 PM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:40,226 PM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-420: Closing 03:04:40,226 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 03:04:40,226 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:40,226 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 03:04:40,227 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:40,227 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 03:04:40,227 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 03:04:40,226 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:40,227 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 03:04:40,228 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 03:04:40,228 PM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 03:04:40,228 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.raft.client.messages.Shutdown] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:40,228 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:40,228 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:40,228 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-420394350] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:40,229 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:40,229 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:40,233 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:40,233 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:40,233 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:40,233 PM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:40,236 PM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:40,250 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 03:04:40,250 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 03:04:40,250 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 03:04:40,250 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 03:04:40,251 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 03:04:40,251 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:40,251 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:41,247 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1702284269399300580] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:41,266 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-7585292480282490916] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:41,267 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:41,267 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [03/07/2024 15:04:41.269] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:41,277 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:41,277 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:41,279 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:41.280] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:41,294 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:04:41,302 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [9182732440525817802] 03:04:41,303 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:41,304 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:41,304 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:41,305 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:41,305 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:41,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:41,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:41,319 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:41,322 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:41,328 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7024912703685169644] 03:04:41,328 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:41,334 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:41,334 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:41,334 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:41,334 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:41,348 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:41,357 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8924999831003021577] 03:04:41,357 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:41,359 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:41,359 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:41,359 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:41,361 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:41,361 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:41,368 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:41,369 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:41,370 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:41,371 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 03:04:41,389 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:41,389 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1189742878] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:41,390 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 03:04:41,391 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 03:04:41,391 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 03:04:41,391 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,391 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:04:41,391 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@33d035ce to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@19a0b77a 03:04:41,391 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@229212c 03:04:41,392 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:41,392 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,392 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1709823881392, None)) 03:04:41,392 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823881391,0,0)) 03:04:41,392 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@19a0b77a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@511a521a 03:04:41,392 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 03:04:41,393 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:41,395 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:41,395 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:41,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@72e953c1 03:04:41,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:41,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,396 PM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,396 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332] 03:04:41,397 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 03:04:41,397 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:41,397 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#157217323 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,397 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#157217323] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:41,397 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-1567332954] 03:04:41,397 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#-1567332954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:41,397 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 03:04:41,399 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 03:04:41,400 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-13175688] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:41,401 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 03:04:41,402 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:41,402 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:41,402 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:41,402 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 03:04:41,402 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 03:04:41,402 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 03:04:41,402 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 03:04:41,402 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 03:04:41,407 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 03:04:41,408 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 03:04:41,408 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 03:04:41,408 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:04:41,408 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,408 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7b527d43 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1c1b4951 03:04:41,409 PM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@35f0d73 03:04:41,409 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:41,409 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,409 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1709823881409, None)) 03:04:41,409 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823881408,0,0)) 03:04:41,409 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2bf0789 03:04:41,409 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1c1b4951 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2af9db72 03:04:41,410 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 03:04:41,410 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:41,410 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,410 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,410 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,410 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#1429898458] 03:04:41,411 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 03:04:41,411 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#1429898458] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:41,411 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-2062825190] 03:04:41,411 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-855380813 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-855380813] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:41,411 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-2062825190] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:41,411 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 03:04:41,412 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:41,412 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:41,412 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 03:04:41,412 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 03:04:41,412 PM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:41,412 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 03:04:41,412 PM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,412 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 03:04:41,412 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 03:04:41,412 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 03:04:41,460 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-1189742878] 03:04:41,460 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 03:04:41,461 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332] 03:04:41,631 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:41,631 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 03:04:41,631 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 03:04:41,631 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,632 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 03:04:41,632 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:41,632 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 03:04:41,632 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 03:04:41,632 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 03:04:41,632 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 03:04:41,632 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 03:04:41,901 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 03:04:41,901 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:41,901 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 03:04:42,30 PM [cluster-test-shard-dispatcher-24] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:42,181 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 03:04:42,181 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,181 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 03:04:42,312 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:42,312 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:42,313 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:42,313 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,313 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,313 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:42,313 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,313 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,313 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,322 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:42,322 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,322 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,322 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:42,322 PM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,322 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,378 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:42,432 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 03:04:42,432 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,432 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 03:04:42,432 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,436 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:42,436 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 03:04:42,436 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:42,436 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:42,436 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 03:04:42,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:42,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,437 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 03:04:42,438 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 03:04:42,438 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 03:04:42,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 03:04:42,439 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,439 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=13] 03:04:42,439 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:42,439 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:42,440 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 03:04:42,440 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 03:04:42,445 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 03:04:42,445 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:42,491 PM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@691e09a0 03:04:42,491 PM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 1610797942, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@3a157c0a} 03:04:42,491 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@691e09a0 changed state from IDLE to TX_OPEN 03:04:42,491 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 03:04:42,491 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:42,491 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:42,491 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-13175688] 03:04:42,491 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} shard 1 03:04:42,492 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:04:42,492 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 03:04:42,492 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:42,492 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2037/0x00000008016a0f70@1bfc1057 03:04:42,492 PM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@691e09a0 changed state from TX_OPEN to IDLE 03:04:42,492 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@691e09a0 readied transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 03:04:42,492 PM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 03:04:42,492 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 03:04:42,492 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@2b64981 03:04:42,492 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:42,492 PM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 03:04:42,492 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=13] 03:04:42,492 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@697ab5c, attempting to connect 03:04:42,494 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} is not yet known 03:04:42,494 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=11410, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:04:42,496 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@581fc4a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4135aad 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55ac6112 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257]}, enqueuedTicks=928897655662} 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4135aad 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2043/0x00000008016a3198@14ec983b 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55ac6112 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=1, protocol=READY}, enqueuedTicks=928897912358} 03:04:42,496 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-423: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=1, protocol=READY}} 03:04:42,497 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 03:04:42,497 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 03:04:42,497 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 03:04:42,497 PM [cluster-test-shard-dispatcher-25] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 03:04:42,498 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-423: Message size: 486 bytes, total slices to send: 1 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-423: Message does not need to be sliced - sending original message 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2102/0x00000008016c6a50@5610dbcb 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55ac6112 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=0, protocol=SIMPLE}, enqueuedTicks=928898102013} 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-423: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=0, protocol=SIMPLE}} 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-423: Message size: 294 bytes, total slices to send: 1 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-423: Message does not need to be sliced - sending original message 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@691e09a0 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4b19ab 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5394bae0 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@22f02a1e, splicing queue 03:04:42,498 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5394bae0 queue spliced 0 messages 03:04:42,499 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 2.391 ms 03:04:42,500 PM [cluster-test-shard-dispatcher-25] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 03:04:42,500 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 03:04:42,500 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@268de5a2 03:04:42,500 PM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#320338257], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:42,500 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#320338257], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#320338257], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 03:04:42,501 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#320338257], modifications=0, protocol=SIMPLE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:42,504 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#320338257], modifications=1, protocol=READY} failed to process}} 03:04:42,504 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=1, protocol=READY}, enqueuedTicks=928897912358, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#320338257], modifications=1, protocol=READY} failed to process}} 03:04:42,504 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#320338257], modifications=1, protocol=READY} failed to process} 03:04:42,504 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#320338257], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:263) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:48) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:108) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 03:04:42,504 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 03:04:42,504 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=0, protocol=SIMPLE}, enqueuedTicks=928898102013, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 03:04:42,504 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#320338257], modifications=0, protocol=SIMPLE} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0} 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@57955e failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$directCommit$4(AbstractProxyTransaction.java:517) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$16(RemoteProxyTransaction.java:509) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#320338257]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@2e1549da 03:04:42,505 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 03:04:42,506 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 03:04:42,506 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 03:04:42,506 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#320338257]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@2a9abc93 03:04:42,506 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55ac6112 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#320338257]}, enqueuedTicks=928911615722} 03:04:42,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: purging transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 03:04:42,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 03:04:42,506 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 03:04:42,507 PM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@57955e is no longer running 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:42,507 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@68abcb45 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@68abcb45 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}]]] 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:42,507 PM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:42,508 PM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard] 03:04:42,508 PM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=13] 03:04:42,508 PM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55ac6112 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@16db21fb, splicing queue 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}]]] 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 2 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@55ac6112 queue spliced 2 messages 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 2 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: refreshing backend for shard 1 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:42,508 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-13175688] 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:42,508 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 03:04:42,509 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 03:04:42,509 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 03:04:42,509 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 1, nextIndex: 2 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2048/0x00000008016a84c0@13737536 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@31218082}, purged=MutableUnsignedLongSet{size=0}} 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 1 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: replication of member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@4e832021 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: finished purging transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Purged transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testChainedTransactionFailureWithSingleShard, matchIndex: 1, lastIndex: 1, isVoting: true 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Follower's log matches - sending ElectionTimeout 03:04:42,509 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:04:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leader transfer complete 03:04:42,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} not found in inflight queue, checking pending queue 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#320338257]}, enqueuedTicks=928897655662, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} not found in inflight queue, checking pending queue 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#320338257]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Commit index set to 1 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#320338257]}, enqueuedTicks=928911615722, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 0 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@4135aad purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 03:04:42,510 PM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@581fc4a purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74} 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5a9d9cac}, purged=MutableUnsignedLongSet{size=0}} 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Setting last applied to 1 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from 0 to 1 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75} 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Purged transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: fakeSnapshot purging log to 0 for term 4 03:04:42,510 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=1 03:04:42,921 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 03:04:42,921 PM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#1703569332]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:04:42,921 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,921 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-425: Closing 03:04:42,922 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 03:04:42,922 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 03:04:42,932 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 03:04:42,932 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@445d3577 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 03:04:42,932 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1327611413]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@f8e4ff1 03:04:42,932 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 03:04:42,932 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 03:04:42,932 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:42,932 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@15fbb38 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:04:42,933 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:42,933 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1327611413], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@15cc4686 03:04:42,933 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 03:04:42,933 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@445d3577 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 03:04:42,933 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1327611413]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@145e20af 03:04:42,933 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 03:04:42,933 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 03:04:42,933 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-904641920], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:42,933 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 03:04:42,933 PM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 03:04:42,933 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:42,933 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:42,938 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:42,938 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:42,938 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:42,938 PM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:42,944 PM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:42,952 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 03:04:42,953 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 03:04:42,953 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 03:04:42,953 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 03:04:42,954 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 03:04:42,954 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:42,954 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:43,948 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [9182732440525817802] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:43,969 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7024912703685169644] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:43,974 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:43,974 PM [cluster-test-akka.actor.default-dispatcher-20] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:43,975 PM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:43.976] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:43,995 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:43,995 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:43,999 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:44.000] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:44,16 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:04:44,23 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6721377317484945973] 03:04:44,24 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:44,27 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:44,27 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:44,27 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:44,27 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:44,41 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:44,41 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:44,41 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:44,44 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:44,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5342729744387920837] 03:04:44,54 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:44,56 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:44,56 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:44,63 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:44,63 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:44,81 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:44,90 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:44,93 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:44,109 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [9073470975040132418] 03:04:44,109 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:44,110 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:44,110 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:44,116 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:44,116 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:44,122 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:44,123 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:44,125 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 03:04:44,130 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1423925496] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:44,131 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 03:04:44,133 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 03:04:44,133 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,133 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:44,133 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,139 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 03:04:44,139 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:04:44,139 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1b0e2d1a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2e4453ec 03:04:44,139 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3e1212b3 03:04:44,139 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1709823884139, None)) 03:04:44,139 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823884138,0,0)) 03:04:44,140 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2e4453ec to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@53e7739c 03:04:44,140 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@33d21f71 03:04:44,140 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 03:04:44,140 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:44,142 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:44,143 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:44,143 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,144 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 03:04:44,145 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,145 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,145 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203] 03:04:44,145 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 03:04:44,146 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 03:04:44,146 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:44,146 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1285306800] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:44,147 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:44,147 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:44,147 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 03:04:44,147 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 03:04:44,147 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 03:04:44,147 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 03:04:44,151 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#684257800] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:44,152 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#1445189617] 03:04:44,153 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:44,153 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1285306800 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,153 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 03:04:44,153 PM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 03:04:44,153 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 03:04:44,154 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#1445189617] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:44,156 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:44,158 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 03:04:44,162 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 03:04:44,162 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:04:44,162 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@592811a8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b0f501 03:04:44,162 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1883f6f7 03:04:44,162 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1709823884162, None)) 03:04:44,162 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 03:04:44,162 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823884161,0,0)) 03:04:44,162 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,162 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b0f501 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4919a034 03:04:44,162 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 03:04:44,162 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:44,162 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,162 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@47099237 03:04:44,163 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:44,163 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,163 PM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,163 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,163 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#1456441997] 03:04:44,163 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 03:04:44,163 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#1456441997] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:44,164 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-1821647767] 03:04:44,164 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1004916533 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,164 PM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-1821647767] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:44,164 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 03:04:44,164 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-1004916533] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:44,165 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:44,165 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:44,165 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 03:04:44,165 PM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 03:04:44,165 PM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 03:04:44,165 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 03:04:44,165 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 03:04:44,165 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,165 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 03:04:44,165 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 03:04:44,213 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1423925496] 03:04:44,213 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 03:04:44,213 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203] 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 03:04:44,413 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 03:04:44,413 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 03:04:44,643 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 03:04:44,643 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,643 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 03:04:44,882 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 03:04:44,882 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:44,882 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 03:04:45,44 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:45,44 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:45,44 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:45,44 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,44 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,44 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:45,44 PM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,44 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,44 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,49 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:45,49 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,49 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,49 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:45,49 PM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,49 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,129 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:45,152 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 03:04:45,153 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,153 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 03:04:45,153 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,154 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:45,154 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 03:04:45,154 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:45,154 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:45,154 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 03:04:45,154 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:45,154 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,154 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 03:04:45,155 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 03:04:45,155 PM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 03:04:45,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 03:04:45,157 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,157 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=13] 03:04:45,157 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:45,157 PM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:45,157 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 03:04:45,157 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 03:04:45,160 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 03:04:45,160 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:45,169 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ab0a111: creating a new snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 03:04:45,169 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 03:04:45,169 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:45,169 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:45,169 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#684257800] 03:04:45,169 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:45,170 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=0} shard 1 03:04:45,170 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 03:04:45,170 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=13] 03:04:45,170 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 03:04:45,170 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:45,170 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@477e04dd 03:04:45,170 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} scheduling timeout in 2 seconds 03:04:45,170 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1b0d52fc, attempting to connect 03:04:45,171 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} is not yet known 03:04:45,171 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=15790, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:04:45,173 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3819c3df instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b5611fc 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2b5611fc 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2089/0x00000008016bf9a8@46525ad5 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1995613239 nanoseconds 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c764a51 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=931575805756} 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ab0a111 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1a876b3b 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1454066 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@55153669, splicing queue 03:04:45,174 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1454066 queue spliced 0 messages 03:04:45,175 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 813.3 μs 03:04:45,176 PM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: allocating new open snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 03:04:45,176 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-427: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 03:04:45,176 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-427: Message size: 235 bytes, total slices to send: 1 03:04:45,176 PM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-427: Message does not need to be sliced - sending original message 03:04:45,177 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 03:04:45,177 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=931575805756, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 03:04:45,177 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0} 03:04:45,178 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 03:04:45,179 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,179 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-430: Closing 03:04:45,179 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 03:04:45,179 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 03:04:45,179 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:45,179 PM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:45,182 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:45,182 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:45,182 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:45,183 PM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:45,194 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 03:04:45,194 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ab0a111: creating a new transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 03:04:45,195 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 03:04:45,195 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:45,195 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2ab0a111 readied transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 03:04:45,195 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@505d4825 03:04:45,195 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:45,195 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c764a51 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], modifications=1, protocol=SIMPLE}, enqueuedTicks=931600816912} 03:04:45,195 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-428: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], modifications=1, protocol=SIMPLE}} 03:04:45,195 PM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-428: Message size: 399 bytes, total slices to send: 1 03:04:45,195 PM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-428: Message does not need to be sliced - sending original message 03:04:45,195 PM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:45,195 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:46,198 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6721377317484945973] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:46,200 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6721377317484945973] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:47,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 03:04:47,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} is empty 03:04:47,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 03:04:47,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:47,190 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 4768559 nanoseconds 03:04:47,219 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:47,219 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], modifications=1, protocol=SIMPLE}, enqueuedTicks=931600816912, sessionId=0, txSequence=1} 03:04:47,219 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 03:04:47,219 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:47,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} directCommit completed 03:04:47,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED}: initiating purge 03:04:47,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} allocated sequence 1 03:04:47,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1899341968]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@b6fca58 03:04:47,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-76661203], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999930917 nanoseconds 03:04:47,220 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c764a51 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1899341968]}, enqueuedTicks=933625803557} 03:04:47,220 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:04:47,220 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:47,221 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 03:04:47,221 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 03:04:47,221 PM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:47,221 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 03:04:47,222 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 03:04:47,222 PM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 03:04:47,222 PM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 03:04:47,222 PM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 03:04:47,223 PM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 03:04:47,223 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:47,224 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:48,239 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5342729744387920837] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:48,243 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:48,243 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:48,243 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:48.245] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:48,251 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:48,251 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:48,253 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:48.254] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:48,264 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:48,284 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1742044625938751443] 03:04:48,285 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:48,287 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:48,287 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:48,287 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:48,287 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:48,300 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:48,300 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:48,300 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:48,303 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 03:04:48,313 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [336628084767247020] 03:04:48,313 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:48,315 PM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:48,315 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:48,328 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:48,328 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:48,345 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:48,353 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7855579631656096648] 03:04:48,353 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:48,355 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:48,355 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:48,361 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:48,361 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:48,367 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:48,367 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:48,368 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:48,369 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:48,370 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 03:04:48,374 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:48,383 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#882921634] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:48,388 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 03:04:48,388 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:04:48,388 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@409a08a0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@569beb32 03:04:48,388 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@fd34b79 03:04:48,388 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:48,388 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,389 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:48,389 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709823888388, None)) 03:04:48,389 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,393 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823888387,0,0)) 03:04:48,393 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@42dc7e2 03:04:48,393 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@569beb32 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@45b9fafc 03:04:48,393 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 03:04:48,394 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:48,395 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:48,397 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 03:04:48,398 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:48,398 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,399 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:48,399 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,399 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,399 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147] 03:04:48,399 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 03:04:48,399 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:48,400 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1371582055] 03:04:48,400 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1473666303 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,400 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1473666303] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:48,400 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1371582055] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:48,400 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 03:04:48,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:48,401 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:48,401 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 03:04:48,401 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 03:04:48,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:48,401 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:48,401 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 03:04:48,401 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 03:04:48,402 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:48,402 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:48,403 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:48,404 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 03:04:48,404 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 03:04:48,405 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:04:48,405 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3ebc335b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@79f4a2ad 03:04:48,405 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:48,405 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,405 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:48,405 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,405 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@fc8ceca 03:04:48,405 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709823888405, None)) 03:04:48,406 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823888404,0,0)) 03:04:48,406 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@79f4a2ad to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5e93cc14 03:04:48,406 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7026f0d8 03:04:48,406 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 03:04:48,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:48,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,407 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,407 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079] 03:04:48,407 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 03:04:48,408 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 03:04:48,408 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:48,408 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#442539804] 03:04:48,408 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-2075322676 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,408 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-2075322676] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:48,408 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#442539804] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:48,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:48,408 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:48,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:48,408 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:48,408 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 03:04:48,408 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 03:04:48,408 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 03:04:48,408 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,409 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 03:04:48,409 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 03:04:48,457 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#882921634] 03:04:48,457 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 03:04:48,457 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147] 03:04:48,692 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:48,692 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:04:48,692 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:48,692 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:48,692 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 03:04:48,692 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:48,692 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:48,693 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:48,693 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:48,693 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:04:48,693 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:04:48,960 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:49,12 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:49,13 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,13 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 03:04:49,304 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:49,304 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:49,304 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:49,304 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,304 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,304 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:49,304 PM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,304 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,304 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,310 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:49,310 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,310 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,310 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:49,310 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,310 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,313 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:49,313 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,313 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 03:04:49,313 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:49,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:49,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:49,314 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:49,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=null 03:04:49,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:49,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,314 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 03:04:49,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 03:04:49,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 03:04:49,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:49,315 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 03:04:49,316 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:04:49,316 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:49,316 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:04:49,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:49,316 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:49,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 03:04:49,316 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 03:04:49,316 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:49,366 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1351dc8c: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:04:49,366 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 03:04:49,366 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:49,366 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:49,366 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:49,366 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} shard 1 03:04:49,366 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:04:49,366 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:49,366 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1351dc8c readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 03:04:49,367 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@4fa112b 03:04:49,367 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:49,367 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} scheduling timeout in 2 seconds 03:04:49,367 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 03:04:49,367 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 03:04:49,367 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4f70acd3, attempting to connect 03:04:49,369 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} is not yet known 03:04:49,369 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=10611, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:04:49,371 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@18279a6 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@544e458d 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@544e458d 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=2, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2102/0x00000008016c6a50@2ca24cbb 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2df670ad transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=2, protocol=SIMPLE}, enqueuedTicks=935772726765} 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-433: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=2, protocol=SIMPLE}} 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-433: Message size: 413 bytes, total slices to send: 1 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-433: Message does not need to be sliced - sending original message 03:04:49,371 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1351dc8c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:49,372 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 03:04:49,372 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@489bac48 03:04:49,372 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4895ff23 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3aec0b04, splicing queue 03:04:49,372 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@4895ff23 queue spliced 0 messages 03:04:49,372 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 883.7 μs 03:04:49,372 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 ready 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transitioned member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 to ready 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct canCommit 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 validated 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct preCommit 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 candidate ready 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - messages: [] 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 prepared 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct commit 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 182 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=182}] 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=182}] 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=182}]]] 03:04:49,373 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 submitted to persistence 03:04:49,374 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=182}] 03:04:49,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=182}]]] 03:04:49,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 03:04:49,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:04:49,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=182} 03:04:49,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=182}] 03:04:49,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 03:04:49,376 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:49,377 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=Simple{size=182}] 03:04:49,377 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@444b73a6}, purged=MutableUnsignedLongSet{size=0}} 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1366713865] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:04:49,377 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 03:04:49,378 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1366713865] 03:04:49,378 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 03:04:49,378 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 108.6 μs 03:04:49,378 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1366713865] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:49,379 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:49,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 03:04:49,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:04:49,379 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=2, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 03:04:49,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 03:04:49,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 03:04:49,379 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 03:04:49,379 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=2, protocol=SIMPLE}, enqueuedTicks=935772726765, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 03:04:49,379 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=2, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0} 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:49,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@27dc809c 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED}: initiating purge 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 1 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@32b091c3 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2df670ad transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]}, enqueuedTicks=935785706151} 03:04:49,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@284b6809}, purged=MutableUnsignedLongSet{size=0}} 03:04:49,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:49,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1630299195] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 03:04:49,380 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 03:04:49,380 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1630299195] 03:04:49,381 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 03:04:49,381 PM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1630299195] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:49,381 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 114.0 μs 03:04:49,381 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,381 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:49,381 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:49,381 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:49,382 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 03:04:49,383 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,383 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 03:04:49,383 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,383 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:49,383 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 03:04:49,383 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:49,383 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:49,384 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,384 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-435: Closing 03:04:49,384 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 03:04:49,384 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:49,385 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:49,385 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:49,389 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:49,389 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:49,389 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:49,389 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:49,390 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:49,404 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 03:04:49,454 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:49,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 03:04:49,454 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079] 03:04:49,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,504 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 4 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:49,510 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:49,510 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:04:49,511 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 03:04:49,511 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:49,511 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2df670ad superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@1a6eea1b, splicing queue 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2df670ad queue spliced 1 messages 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:49,511 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:49,511 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:49,511 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 03:04:49,560 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:49,569 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6356741021260016236] 03:04:49,570 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:49,571 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:49,571 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:49,578 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:49,578 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:49,583 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:49,584 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:49,586 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 03:04:49,599 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#2144642113] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:49,604 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3819c3df aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 03:04:49,604 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:04:49,604 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@5793f94b 03:04:49,604 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} has a successor, not scheduling timer 03:04:49,604 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SEALED} allocated sequence 1 03:04:49,604 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2107/0x00000008016ccb68@2a1fc58f 03:04:49,604 PM [Cleaner-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c764a51 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], modifications=0, protocol=ABORT}, enqueuedTicks=936010309082} 03:04:49,604 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-428: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1899341968], modifications=0, protocol=ABORT}} 03:04:49,604 PM [Cleaner-1] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-428: Message size: 294 bytes, total slices to send: 1 03:04:49,604 PM [Cleaner-1] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-428: Message does not need to be sliced - sending original message 03:04:49,604 PM [Cleaner-1] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0} 03:04:49,610 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:49,612 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:04:49,613 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 03:04:49,613 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4f98e2f8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2c9ec4fe 03:04:49,613 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@699c5f09 03:04:49,613 PM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1709823889613, None)) 03:04:49,613 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823889612,0,0)) 03:04:49,613 PM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:49,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,613 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2c9ec4fe to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7e81cc3e 03:04:49,613 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@40bc6607 03:04:49,614 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 03:04:49,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:49,614 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,614 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 03:04:49,614 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:49,615 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1229499622 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,615 PM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 03:04:49,615 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1229499622] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:49,618 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1478313394] 03:04:49,618 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1478313394] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:49,618 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:04:49,618 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,618 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:04:49,618 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,618 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 182 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@78b131ec}, purged=MutableUnsignedLongSet{size=0}} 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 74 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:49,619 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 700.1 μs - Switching actor to Follower - last log index = 1, last log term = 3, snapshot index = -1, snapshot term = -1, journal size = 2 03:04:49,620 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 03:04:49,620 PM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 03:04:49,621 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:49,622 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:49,622 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 03:04:49,622 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 03:04:49,622 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 03:04:49,664 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:49,664 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 03:04:49,664 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079] 03:04:49,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:49,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,720 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 03:04:49,720 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,730 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 03:04:49,730 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 03:04:49,730 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:49,730 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:49,731 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:49,731 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:49,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:49,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:49,930 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 03:04:49,930 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,79 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:50,79 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,79 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 03:04:50,79 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 4 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:50,188 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:04:50,188 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:04:50,300 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:50,300 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,300 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 03:04:50,300 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,399 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1742044625938751443] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:50,400 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [1742044625938751443] but current is [6356741021260016236]. Reason: ActorSystem terminated 03:04:50,424 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3], my term is 4 03:04:50,424 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than Candidate's term 4 - switching to Follower 03:04:50,424 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=null 03:04:50,425 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] LocalActorRefProvider$$anon$2 - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [336628084767247020]. This system with UID [6356741021260016236] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:50,425 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:50,425 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=5, voteGranted=false] 03:04:50,425 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 5 03:04:50,425 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:50,425 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:50,425 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 03:04:50,425 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 03:04:50,426 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=false], my term is 8 03:04:50,426 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=false], current voteCount: 1 03:04:50,426 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 03:04:50,426 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 5 - updating term 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=false] 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 6 - updating term 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=false] 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:50,427 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:50,428 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:50,428 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:50,428 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 7 - updating term 03:04:50,428 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 03:04:50,428 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:50,428 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=false] 03:04:50,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=false], my term is 8 03:04:50,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=false], current voteCount: 1 03:04:50,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=7, voteGranted=false], my term is 8 03:04:50,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=7, voteGranted=false], current voteCount: 1 03:04:50,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=8, voteGranted=false], my term is 8 03:04:50,428 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=8, voteGranted=false], current voteCount: 1 03:04:50,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:50,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,451 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 03:04:50,451 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,453 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:50,453 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:50,453 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:50,453 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 8 - updating term 03:04:50,453 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 03:04:50,453 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:50,454 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=false] 03:04:50,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=9, voteGranted=false], my term is 9 03:04:50,454 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=9, voteGranted=false], current voteCount: 1 03:04:50,474 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#365824037]] intended for system uid [1742044625938751443], self uid is [6356741021260016236] 03:04:50,590 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 03:04:50,650 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:50,650 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,650 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 03:04:50,650 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,651 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:50,651 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:50,651 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:50,651 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 9 - updating term 03:04:50,652 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 03:04:50,652 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:50,652 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=false] 03:04:50,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=10, voteGranted=false], my term is 10 03:04:50,652 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=10, voteGranted=false], current voteCount: 1 03:04:50,750 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:50,750 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:50,750 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:50,750 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:50,750 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:50,750 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 03:04:50,830 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:50,830 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,830 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 11 03:04:50,830 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:50,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:50,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:50,831 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 11 in "RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 10 - updating term 03:04:50,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=null 03:04:50,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 11, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:50,831 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=11, voteGranted=false] 03:04:50,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=11, voteGranted=false], my term is 11 03:04:50,832 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=11, voteGranted=false], current voteCount: 1 03:04:50,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 03:04:50,969 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 03:04:50,969 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:50,970 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:50,970 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:50,970 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:50,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:50,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,979 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 12 03:04:50,979 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:50,981 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:50,981 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:50,981 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:50,981 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 12 in "RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 11 - updating term 03:04:50,981 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=null 03:04:50,981 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 12, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:50,981 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=12, voteGranted=false] 03:04:50,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=12, voteGranted=false], my term is 12 03:04:50,982 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=12, voteGranted=false], current voteCount: 1 03:04:51,27 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 03:04:51,27 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:04:51,28 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:51,28 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,28 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 13 03:04:51,28 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 13 03:04:51,28 PM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:51,28 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:51,28 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:51,28 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:04:51,28 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:04:51,139 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:51,139 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,139 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 13 03:04:51,140 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,141 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3], my term is 13 03:04:51,141 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 13, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 3 03:04:51,141 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=13, voteGranted=false] 03:04:51,142 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=13, voteGranted=false], my term is 13 03:04:51,142 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=13, voteGranted=false], current voteCount: 1 03:04:51,279 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:51,279 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,279 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 14 03:04:51,280 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3], my term is 13 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 14 in "RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than Candidate's term 13 - switching to Follower 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=null 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 14, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=14, voteGranted=false] 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 14 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:51,281 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 03:04:51,281 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 03:04:51,282 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=14, voteGranted=false], my term is 14 03:04:51,282 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=14, voteGranted=false], current voteCount: 1 03:04:51,379 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 03:04:51,379 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} is empty 03:04:51,379 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 03:04:51,389 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:51,389 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 03:04:51,389 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:51,420 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 03:04:51,420 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [1742044625938751443] but current is [6356741021260016236]. Reason: Cluster member removed, previous status [Down] 03:04:51,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:51,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,484 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 15 03:04:51,484 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:51,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:51,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:51,485 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 15 in "RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 14 - updating term 03:04:51,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=null 03:04:51,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 15, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:51,485 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=15, voteGranted=false] 03:04:51,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=15, voteGranted=false], my term is 15 03:04:51,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=15, voteGranted=false], current voteCount: 1 03:04:51,530 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:51,530 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 03:04:51,609 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 03:04:51,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:51,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,640 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 16 03:04:51,640 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,641 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:51,641 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:51,641 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:51,641 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 16 in "RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 15 - updating term 03:04:51,641 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=null 03:04:51,641 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 16, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:51,642 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=16, voteGranted=false] 03:04:51,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=16, voteGranted=false], my term is 16 03:04:51,642 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=16, voteGranted=false], current voteCount: 1 03:04:51,741 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [1742044625938751443] but current is [6356741021260016236]. Reason: Cluster member removed, previous status [Down] 03:04:51,741 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberRemoved: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:51,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:51,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,770 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 17 03:04:51,770 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,771 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:51,771 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:51,771 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:51,771 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 17 in "RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 16 - updating term 03:04:51,771 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=null 03:04:51,771 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 17, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:51,771 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=17, voteGranted=false] 03:04:51,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=17, voteGranted=false], my term is 17 03:04:51,771 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=17, voteGranted=false], current voteCount: 1 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 18 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 18 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:51,898 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:04:51,898 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:04:51,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:51,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,919 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 18 03:04:51,919 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:51,920 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3], my term is 18 03:04:51,920 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=18, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 18, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 3 03:04:51,920 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=18, voteGranted=false] 03:04:51,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=18, voteGranted=false], my term is 18 03:04:51,921 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=18, voteGranted=false], current voteCount: 1 03:04:51,990 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:51,990 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:51,990 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:51,990 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:51,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:51,990 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 03:04:52,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:52,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,110 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 19 03:04:52,110 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3], my term is 18 03:04:52,111 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 19 in "RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than Candidate's term 18 - switching to Follower 03:04:52,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=null 03:04:52,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 19, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:52,111 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=19, voteGranted=false] 03:04:52,112 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 19 03:04:52,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:52,112 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:52,112 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 03:04:52,112 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 03:04:52,112 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=19, voteGranted=false], my term is 19 03:04:52,112 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=19, voteGranted=false], current voteCount: 1 03:04:52,210 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 03:04:52,210 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 03:04:52,210 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:52,210 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:52,210 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:52,211 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:52,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:52,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,239 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 20 03:04:52,239 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,240 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:52,240 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:52,241 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:52,241 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 20 in "RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 19 - updating term 03:04:52,241 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=null 03:04:52,241 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 20, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:52,241 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=20, voteGranted=false] 03:04:52,241 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=20, voteGranted=false], my term is 20 03:04:52,241 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=20, voteGranted=false], current voteCount: 1 03:04:52,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:52,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=21, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,439 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 21 03:04:52,439 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,440 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:52,441 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:52,441 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:52,441 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 21 in "RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 20 - updating term 03:04:52,441 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=21, votedFor=null 03:04:52,441 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=21, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 21, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:52,441 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=21, voteGranted=false] 03:04:52,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=21, voteGranted=false], my term is 21 03:04:52,441 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=21, voteGranted=false], current voteCount: 1 03:04:52,569 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 03:04:52,569 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=22, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,570 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 22 03:04:52,570 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=22, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:52,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 03:04:52,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=22, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] with higher term appears reachable, updating term. 03:04:52,571 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 22 in "RequestVote [term=22, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3]" message is greater than follower's term 21 - updating term 03:04:52,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=22, votedFor=null 03:04:52,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=22, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=3] - currentTerm: 22, votedFor: null, lastIndex: 1, lastTerm: 3 03:04:52,571 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=22, voteGranted=false] 03:04:52,572 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=22, voteGranted=false], my term is 22 03:04:52,572 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=22, voteGranted=false], current voteCount: 1 03:04:52,629 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING, roles [member-1, dc-default], version [0.0.0] 03:04:52,631 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Welcome from [akka://cluster-test@127.0.0.1:2559] 03:04:52,632 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:52,632 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,632 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,632 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:52,632 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,632 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,632 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,677 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Joining), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Joining) 03:04:52,677 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 03:04:52,677 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:52,677 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=23, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,678 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 23 03:04:52,678 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=23, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,678 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 23 03:04:52,678 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:52,678 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:52,678 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:52,678 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:04:52,678 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=23, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3], my term is 22 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 23 in "RequestVote [term=23, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3]" message is greater than Candidate's term 22 - switching to Follower 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=23, votedFor=null 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=23, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=3] - currentTerm: 23, votedFor: null, lastIndex: 0, lastTerm: 3 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=23, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=23, voteGranted=true] 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 23 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:52,679 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:52,679 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 03:04:52,680 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=23, voteGranted=true], my term is 23 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=23, voteGranted=true], current voteCount: 1 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): LastApplied index 0 is behind last index 1 - switching to PreLeader 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 23 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=23, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=23, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:52,680 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Replicate message: identifier: null, logIndex: 2, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 03:04:52,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:52,681 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to PreLeader 03:04:52,681 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:52,681 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,681 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:04:52,681 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:52,681 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 03:04:52,681 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:52,681 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:04:52,681 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=23, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:52,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:52,681 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:52,681 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to PreLeader 03:04:52,682 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 03:04:52,682 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 03:04:52,682 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 03:04:52,682 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,682 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=23, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:04:52,683 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=23, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2]]] 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 2 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=23, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 3 03:04:52,683 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=23, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=23, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:52,684 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:52,684 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=23, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2] 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 3 does not match the current term 23 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 1 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 has no callback 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 2 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 23 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Received ApplyState [identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=3, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}]] - lastApplied: 2, lastIndex: 2 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 23 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader)_slicer-id-438: Closing 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 03:04:52,685 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from PreLeader to Leader 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from PreLeader to Leader 03:04:52,685 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 2 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 2 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@6c33a0a2 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 23 03:04:52,686 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 03:04:52,722 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:52,722 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1351dc8c: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:04:52,722 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:04:52,722 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:52,723 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1351dc8c readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 03:04:52,723 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@26dd8bef 03:04:52,723 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 03:04:53,229 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:53,229 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:53,229 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:53,229 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:53,230 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:53,230 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, activeMember=true] 03:04:53,450 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 03:04:53,450 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 03:04:53,450 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:53,450 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:53,450 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:53,451 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:53,459 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:53,469 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:53,469 PM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:53,469 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:53,469 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:04:53,550 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:53,550 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 03:04:53,651 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:53,651 PM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:53,651 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:53,651 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:04:53,651 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,470 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:54,470 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:54,470 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:54,470 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:54,470 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:54,470 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 03:04:54,470 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=13] 03:04:54,471 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@64ecadaf, attempting to connect 03:04:54,473 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:04:54,473 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:54,473 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:54,473 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@544e458d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393e233 03:04:54,473 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state DONE 03:04:54,473 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 03:04:54,473 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393e233 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE}: initiating purge 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 0 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@380bae46 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@367efc3b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423]}, enqueuedTicks=935785706151} 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393e233 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=1, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2102/0x00000008016c6a50@629a4e6b 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@367efc3b transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=1, protocol=SIMPLE}, enqueuedTicks=939128859260} 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-433: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=1, protocol=SIMPLE}} 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-433: Message size: 459 bytes, total slices to send: 1 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-433: Message does not need to be sliced - sending original message 03:04:54,474 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1351dc8c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:54,475 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 03:04:54,475 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} 03:04:54,475 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5ba7866 03:04:54,475 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@578650bf superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7b489e59, splicing queue 03:04:54,475 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@578650bf queue spliced 0 messages 03:04:54,475 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.363 ms 03:04:54,478 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:54,478 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=3, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:54,479 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:54,479 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:54,479 PM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,479 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 ready 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transitioned member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 to ready 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct canCommit 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 validated 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct preCommit 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 candidate ready 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - candidate rootPath: / 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - messages: [] 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 prepared 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct commit 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=4, term=23, payload=Simple{size=241}] 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=23, payload=Simple{size=241}] 03:04:54,480 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:04:54,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 03:04:54,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=23, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=4, term=23, payload=Simple{size=241}]]] 03:04:54,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 submitted to persistence 03:04:54,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:54,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,481 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=4, term=23, payload=Simple{size=241}] 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=23, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=4, term=23, payload=Simple{size=241}]]] 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 2 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=241} 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=23, payload=Simple{size=241}] 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 3 03:04:54,482 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=23, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=4, logLastTerm=23, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 4, nextIndex: 5 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 3 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2129/0x00000008016e2000@2a243706 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: finished purging inherited transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 4 03:04:54,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 3 to 4 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 4 data Simple{size=241} 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,483 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=4, term=23, payload=Simple{size=241}] 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: commit - successfulFromPrevious: [] 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 03:04:54,483 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2011600225] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 03:04:54,483 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2011600225] 03:04:54,484 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 3 for term 23 03:04:54,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 03:04:54,484 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:04:54,484 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=4 03:04:54,484 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 105.6 μs 03:04:54,484 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#2011600225] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 4 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 3 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423]}, enqueuedTicks=935785706151, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE}: purge completed 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393e233 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@544e458d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 4 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 3 to 4 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@18279a6 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 4 data Simple{size=241} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=1, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=1, protocol=SIMPLE}, enqueuedTicks=939128859260, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=1, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@58e72c38 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1497497147], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED}: initiating purge 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} allocated sequence 1 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@e48060a 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:54,485 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@367efc3b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]}, enqueuedTicks=940891214512} 03:04:54,485 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1351dc8c: creating a new snapshot member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 03:04:54,485 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 03:04:54,485 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} allocated sequence 0 03:04:54,485 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:54,486 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@68fa0c6f 03:04:54,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,486 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@367efc3b transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=940891687934} 03:04:54,486 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 03:04:54,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 3 for term 23 03:04:54,486 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=4 03:04:54,486 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 72.38 μs 03:04:54,487 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,487 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=5, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 03:04:54,487 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 03:04:54,487 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:04:54,487 PM [cluster-test-shard-dispatcher-16] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open snapshot member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 03:04:54,487 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-437: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, message: Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 03:04:54,487 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-437: Message size: 378 bytes, total slices to send: 1 03:04:54,487 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges_slicer-id-437: Message does not need to be sliced - sending original message 03:04:54,488 PM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=5, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 03:04:54,489 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 03:04:54,489 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=940891687934, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0}} 03:04:54,489 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0} 03:04:54,489 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 03:04:54,489 PM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,489 PM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 03:04:54,489 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 03:04:54,489 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Initiating leader transfer 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@64ce3165 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@64ce3165 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Attempting to transfer leadership 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=4, prevLogTerm=23, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}]]] 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=13] 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:54,490 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:54,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:54,490 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=13] 03:04:54,490 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@367efc3b superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@6b2eacd4, splicing queue 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@367efc3b queue spliced 1 messages 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:54,491 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#144883957] 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=4, prevLogTerm=23, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}]]] 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=23, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=5, logLastTerm=23, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:54,491 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:04:54,492 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=5, term=23, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 5, nextIndex: 6 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 5 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 4 to 5 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@59d642cb 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: finished purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 4 for term 23 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, matchIndex: 5, lastIndex: 5, isVoting: true 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Follower's log matches - sending ElectionTimeout 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Leader transfer complete 03:04:54,492 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=5 03:04:54,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} not found in inflight queue, checking pending queue 03:04:54,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} 03:04:54,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423]}, enqueuedTicks=940891214512, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1}} 03:04:54,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE}: purge completed 03:04:54,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393e233 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} 03:04:54,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 03:04:54,493 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@544e458d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 5 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 5 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 4 to 5 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75} 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 4 for term 23 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received TimeoutNow - switching to Candidate 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:54,493 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=24, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 24 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=24, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=23] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 24 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:54,494 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 03:04:54,494 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 03:04:54,494 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=5 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Cluster state: [] 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Candidate in requestVote:RequestVote [term=24, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=23] with higher term appears reachable, updating term. 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Term 24 in "RequestVote [term=24, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=23]" message is greater than leader's term 23 - switching to Follower 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=24, votedFor=null 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Leadership transfer in progress - processing RequestVote 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): In requestVote: RequestVote [term=24, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=5, lastLogTerm=23] - currentTerm: 24, votedFor: null, lastIndex: 5, lastTerm: 23 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=24, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,494 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): requestVote returning: RequestVoteReply [term=24, voteGranted=true] 03:04:54,495 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader) :- Switching from behavior Leader to Follower, election term: 24 03:04:54,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-439: Closing 03:04:54,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 03:04:54,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 03:04:54,495 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:54,495 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Leader to Follower 03:04:54,495 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 03:04:54,495 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Leader to Follower 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=24, voteGranted=true], my term is 24 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=24, voteGranted=true], current voteCount: 1 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 4 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 24 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=13] 03:04:54,495 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 03:04:54,495 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized - notifying 1 OnShardInitialized callbacks 03:04:54,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:54,496 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 03:04:54,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is ready - notifying 1 OnShardInitialized callbacks 03:04:54,496 PM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7be08996] 03:04:54,496 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 03:04:54,496 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:04:54,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:54,496 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7be08996] 03:04:54,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,496 PM [cluster-test-shard-dispatcher-27] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Successfully transferred leadership to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges in 6.432 ms 03:04:54,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader transfer succeeded - sending PoisonPill 03:04:54,496 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@13408a42, attempting to connect 03:04:54,496 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 03:04:54,496 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1229499622] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1229499622] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:54,496 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 03:04:54,496 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:54,496 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 03:04:54,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 5, nextIndex: 6 03:04:54,496 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393e233 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@14b8949 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@14b8949 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423], snapshotOnly=true} 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@67270ed4 to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 replayed 1 successful requests 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1351dc8c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@62b267c4 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7fea451d superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@22b69a68, splicing queue 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7fea451d queue spliced 0 messages 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 381.1 μs 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 03:04:54,497 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:54,497 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:54,498 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:54,498 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:04:54,498 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 03:04:54,498 PM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:54,498 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:54,498 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:54,498 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 03:04:54,498 PM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:405) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 03:04:54,499 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:54,498 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#2144642113] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#2144642113] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:54,499 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-1943593498] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1943593498] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:54,499 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:54,499 PM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:54,503 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:54,503 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:54,503 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:54,503 PM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:54,505 PM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:04:54,520 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 03:04:54,520 PM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 03:04:54,521 PM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-440: Closing 03:04:54,521 PM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 03:04:54,521 PM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 03:04:54,521 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:54,521 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:55,519 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6356741021260016236] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:55,539 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [336628084767247020] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:04:55,541 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:55,542 PM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:04:55,543 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:55.545] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:55,551 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:04:55,551 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:04:55,553 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:04:55.554] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:55,567 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:55,572 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4063228306069740001] 03:04:55,572 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:04:55,574 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:04:55,574 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:04:55,577 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:55,577 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:55,585 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:04:55,585 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:04:55,585 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:04:55,587 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:55,592 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2677380883864797108] 03:04:55,592 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:04:55,594 PM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:55,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:04:55,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:55,594 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:55,613 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:04:55,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2974973494404331377] 03:04:55,620 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:04:55,622 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:04:55,622 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:04:55,623 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:04:55,623 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:04:55,632 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:55,633 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:55,635 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 03:04:55,643 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:04:55,644 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:55,645 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 03:04:55,648 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 03:04:55,648 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,648 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 03:04:55,648 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:04:55,648 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@174ef159 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f72099a 03:04:55,648 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@269347fd 03:04:55,648 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:55,648 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,648 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709823895648, None)) 03:04:55,648 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823895647,0,0)) 03:04:55,649 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f72099a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@57a6f46 03:04:55,649 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@584f88a4 03:04:55,649 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 03:04:55,649 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:55,650 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:55,650 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,651 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:04:55,652 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:55,653 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 03:04:55,653 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,653 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] 03:04:55,654 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 03:04:55,654 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 03:04:55,655 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:55,655 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#1391583425] 03:04:55,655 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#632629054 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,655 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#632629054] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:55,657 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:55,657 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:55,657 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 03:04:55,657 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:55,657 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 03:04:55,657 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 03:04:55,657 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#1391583425] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:55,657 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 03:04:55,657 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 03:04:55,658 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 03:04:55,658 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 03:04:55,661 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1267953125] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:55,664 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 03:04:55,664 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:04:55,664 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@19866d1d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1eb5b977 03:04:55,664 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6189bbdf 03:04:55,664 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 03:04:55,664 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:55,664 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709823895664, None)) 03:04:55,664 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823895663,0,0)) 03:04:55,664 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:04:55,664 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,664 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@24d0a490 03:04:55,665 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1eb5b977 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@30fc7103 03:04:55,665 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 03:04:55,665 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:04:55,665 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:55,665 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,665 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:55,665 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#2111065821] 03:04:55,665 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 03:04:55,665 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#2111065821] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:55,666 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#428663197 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:55,666 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#428663197] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:55,666 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-1596142215] 03:04:55,666 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-1596142215] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:55,666 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 03:04:55,667 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:04:55,667 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:04:55,667 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:55,667 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 03:04:55,667 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 03:04:55,667 PM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 03:04:55,667 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,667 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 03:04:55,667 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 03:04:55,667 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 03:04:55,678 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:04:55,679 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:04:55,715 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:04:55,715 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 03:04:55,715 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:04:55,963 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 03:04:55,964 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 03:04:55,964 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 03:04:56,266 PM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:04:56,282 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 03:04:56,282 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:56,283 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 03:04:56,593 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 03:04:56,593 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:56,593 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 03:04:56,594 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:04:56,594 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:04:56,595 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:56,595 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 03:04:56,595 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:56,595 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:56,595 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:56,595 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:56,595 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:56,602 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:04:56,602 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 03:04:56,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:56,602 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:04:56,602 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:56,602 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:56,635 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:04:56,853 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 03:04:56,853 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:56,853 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 03:04:56,853 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:56,855 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:04:56,855 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 03:04:56,855 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:04:56,855 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:04:56,855 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 03:04:56,855 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:04:56,855 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:56,855 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 03:04:56,856 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 03:04:56,856 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 03:04:56,856 PM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 03:04:56,856 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:56,857 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 03:04:56,857 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 03:04:56,857 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:04:56,857 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:04:56,857 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 03:04:56,857 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 03:04:56,857 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:56,879 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@777d7ac1: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:04:56,880 PM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} 03:04:56,880 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:56,880 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:56,880 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:04:56,880 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:56,880 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} shard 1 03:04:56,880 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:04:56,880 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@45b74e13] 03:04:56,880 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:04:56,880 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@777d7ac1: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 03:04:56,880 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 03:04:56,880 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:04:56,880 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@777d7ac1: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:04:56,880 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@45b74e13] 03:04:56,880 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:04:56,880 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} allocated sequence 0 03:04:56,880 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@267cc758, attempting to connect 03:04:56,880 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} is not yet known 03:04:56,881 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=9960, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:04:56,881 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@509c76aa instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@34d3a9b9 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state OPEN 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@34d3a9b9 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@34d3a9b9 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@34d3a9b9 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@777d7ac1 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7a3a35c6 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@674849fa superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@54d24a9f, splicing queue 03:04:56,881 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@674849fa queue spliced 0 messages 03:04:56,882 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 596.8 μs 03:04:56,891 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:04:56,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:56,891 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 03:04:56,891 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 03:04:56,892 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:04:56,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 03:04:56,892 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 03:04:56,897 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 03:04:56,897 PM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 03:04:56,897 PM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:04:56,897 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 03:04:56,897 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:04:56,897 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], modifications=1} 03:04:56,897 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1931/0x0000000801657020] to Actor[akka://cluster-test/user/$a#-340116130] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-340116130] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:56,898 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1267953125] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1267953125] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:56,898 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1267953125] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1267953125] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:56,898 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@777d7ac1 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 03:04:56,898 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 03:04:56,899 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:04:56,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 03:04:56,899 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 03:04:56,899 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 03:04:56,899 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] 03:04:56,899 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] 03:04:56,950 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] 03:04:56,974 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:57,0 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] 03:04:57,51 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] 03:04:57,93 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 03:04:57,93 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 03:04:57,93 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:04:57,94 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 03:04:57,94 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-445: Closing 03:04:57,94 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:04:57,94 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 03:04:57,94 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 03:04:57,94 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:57,95 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:04:57,101 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] 03:04:57,102 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], modifications=1} 03:04:57,102 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@777d7ac1 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 03:04:57,102 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 03:04:57,102 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@b2c8c37 03:04:57,102 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:57,102 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4937e5d1 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}, enqueuedTicks=943507938249} 03:04:57,102 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: false, paused: false 03:04:57,102 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:57,102 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:627) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4937e5d1 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@7de5afdd, splicing queue 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4937e5d1 queue spliced 1 messages 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:57,103 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:04:57,103 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:57,103 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:04:57,523 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:04:57,523 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:04:57,523 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:57,524 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:57,524 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:57,524 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:57,613 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 03:04:58,543 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:58,543 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:58,543 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:58,543 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:04:58,543 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:58,543 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:04:58,962 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:04:58,962 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:04:58,963 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:04:58,963 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:58,963 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:58,964 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:04:59,982 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:04:59,982 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:04:59,982 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:04:59,982 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:04:59,983 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:04:59,983 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:05:00,403 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:05:00,403 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:05:00,404 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:05:00,404 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:00,404 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:00,405 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:01,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:01,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:01,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:01,423 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:05:01,423 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:01,423 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:05:01,843 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:05:01,843 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:05:01,843 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:05:01,843 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:01,843 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:01,844 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:02,863 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:02,863 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:02,863 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:02,863 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:05:02,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:02,863 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:05:03,282 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:05:03,282 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:05:03,283 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:05:03,283 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:03,283 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:03,284 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:04,303 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:04,303 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:04,303 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:04,303 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:05:04,303 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:04,303 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:05:04,723 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:05:04,723 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:05:04,723 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:05:04,723 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:04,724 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:04,724 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:05,743 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:05,743 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:05,743 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:05,743 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:05:05,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:05,743 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:05:06,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:05:06,163 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:05:06,163 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:05:06,163 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:06,163 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:06,163 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:07,183 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:07,183 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:07,183 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:07,183 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:05:07,183 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:07,183 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:05:07,603 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:05:07,603 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:05:07,603 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:05:07,603 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:07,603 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:07,604 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:08,623 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:08,623 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:08,623 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:08,623 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:05:08,623 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:08,623 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 03:05:09,43 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 03:05:09,43 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 03:05:09,43 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 03:05:09,43 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:09,43 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:09,43 PM [ForkJoinPool.commonPool-worker-3] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:649) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:233) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 03:05:09,103 PM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 03:05:09,103 PM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], modifications=1} 03:05:09,103 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@777d7ac1 readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 03:05:09,103 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:05:09,104 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 03:05:09,104 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 03:05:09,106 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:05:09,108 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 03:05:09,111 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 03:05:09,112 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:05:09,113 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 03:05:09,113 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@d05ae4e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6471941 03:05:09,113 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4c3445e2 03:05:09,113 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1709823909113, None)) 03:05:09,113 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823909112,0,0)) 03:05:09,113 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 03:05:09,113 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:09,113 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6471941 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@135604ec 03:05:09,114 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@24b91e08 03:05:09,114 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} allocated sequence 0 03:05:09,114 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@6a49ea8 03:05:09,114 PM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:05:09,114 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-709995850] 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-709995850] 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:09,115 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-709995850] 03:05:09,115 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 03:05:09,115 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 03:05:09,115 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-1913172904 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:09,115 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-584910072] 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:05:09,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 03:05:09,121 PM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 03:05:09,121 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:05:09,121 PM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 03:05:09,121 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 03:05:09,214 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 03:05:09,214 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 03:05:09,214 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 03:05:09,214 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=13] 03:05:09,214 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:05:09,214 PM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:05:09,215 PM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 03:05:09,215 PM [cluster-test-shard-dispatcher-27] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 12357, lastApplied : -1, commitIndex : -1 03:05:09,215 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 03:05:09,215 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 03:05:09,215 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:05:09,215 PM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 03:05:09,215 PM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-446: Closing 03:05:09,215 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 03:05:09,216 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 03:05:09,216 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 03:05:09,216 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 03:05:09,216 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 03:05:10,63 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,63 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:10,63 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:10,63 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:05:10,63 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:10,63 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@45b74e13] 03:05:10,63 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@45b74e13] 03:05:10,63 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@eccd1ed, attempting to connect 03:05:10,64 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@34d3a9b9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@70eca39f 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state SEALED 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@70eca39f 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 03:05:10,64 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 03:05:10,65 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 0 03:05:10,65 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@6a49ea8 03:05:10,65 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@526e7b0 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}, enqueuedTicks=955520153146} 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@70eca39f 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct canCommit 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} allocated sequence 0 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 validated 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@b2c8c37 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct preCommit 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@526e7b0 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}, enqueuedTicks=943507938249} 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@70eca39f 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 candidate ready 03:05:10,66 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - messages: [] 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 prepared 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct commit 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 171 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 03:05:10,66 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 submitted to persistence 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 initiating direct canCommit 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [WARN] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Store Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1: Conflicting modification for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars. 03:05:10,67 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to FAILED 03:05:10,68 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onFailure(FrontendReadWriteTransaction.java:441) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.failedCanCommit(SimpleShardDataTreeCohort.java:175) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:862) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:884) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:831) at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:986) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.directCommit(FrontendReadWriteTransaction.java:433) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleCommitLocalTransaction(FrontendReadWriteTransaction.java:524) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:193) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:633) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:436) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:365) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: OptimisticLockFailedException{message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, errorList=[RpcError [message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:846) ... 41 more Caused by: org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction. at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:74) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:181) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:111) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:409) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:368) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:110) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:31) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:838) ... 41 more 03:05:10,68 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], modifications=1} 03:05:10,68 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@777d7ac1 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,68 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@f39bd65 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7a5efdfd superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@76a399c6, splicing queue 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7a5efdfd queue spliced 0 messages 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 4.679 ms 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}, enqueuedTicks=943507938249, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@f52bf00 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED}: initiating purge 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@4d42e91a 03:05:10,69 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@526e7b0 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]}, enqueuedTicks=956475131099} 03:05:10,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 03:05:10,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 03:05:10,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 03:05:10,69 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:05:10,70 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 03:05:10,70 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 03:05:10,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 03:05:10,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 03:05:10,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:05:10,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=171} 03:05:10,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 03:05:10,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 1 03:05:10,74 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 0, nextIndex: 1 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 0 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@68b2f19a}, purged=MutableUnsignedLongSet{size=0}} 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#2144501814] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}, enqueuedTicks=955520153146, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#2144501814] 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@35389f48 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 03:05:10,75 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:05:10,75 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:10,76 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} 03:05:10,76 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}]]] 03:05:10,76 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED}: initiating purge 03:05:10,76 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} allocated sequence 1 03:05:10,76 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@65f35575 03:05:10,76 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 03:05:10,76 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@526e7b0 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]}, enqueuedTicks=956481513328} 03:05:10,76 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 110.0 μs 03:05:10,76 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,76 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 03:05:10,76 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 03:05:10,76 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#2144501814] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:10,76 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:05:10,76 PM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} allocated sequence 0 03:05:10,77 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2069/0x00000008016b6ed8@1aca2855 03:05:10,77 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@526e7b0 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}, enqueuedTicks=956482749810} 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct canCommit 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 validated 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct preCommit 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 candidate ready 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - candidate rootPath: / 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - messages: [] 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 prepared 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct commit 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 69 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 submitted to persistence 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 03:05:10,77 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 03:05:10,78 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}]]] 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 0 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 03:05:10,79 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,80 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@35e15a84}, purged=MutableUnsignedLongSet{size=0}} 03:05:10,80 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,80 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,80 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1746645023] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 03:05:10,80 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-1746645023] 03:05:10,80 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 03:05:10,80 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 182.8 μs 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 1, nextIndex: 2 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 1 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@5f1a4e9e 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 03:05:10,81 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 03:05:10,81 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]}, enqueuedTicks=956475131099, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 03:05:10,81 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE}: purge completed 03:05:10,81 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@70eca39f purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 03:05:10,81 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:10,81 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@34d3a9b9 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}]]] 03:05:10,81 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 03:05:10,82 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 03:05:10,82 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}]]] 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=69} 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 4 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 1 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 3 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 03:05:10,83 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 03:05:10,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 3, nextIndex: 4 03:05:10,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 03:05:10,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 03:05:10,83 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@234a8182 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 3 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data Simple{size=69} 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]}, enqueuedTicks=956481513328, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE}: purge completed 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@70eca39f purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: commit - successfulFromPrevious: [] 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@34d3a9b9 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1505457766], coordinated=false}, enqueuedTicks=956482749810, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} directCommit completed 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED}: initiating purge 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} allocated sequence 1 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@77610e16 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@526e7b0 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]}, enqueuedTicks=956490068962} 03:05:10,84 PM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 24.97 μs 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 03:05:10,84 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:05:10,85 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:05:10,85 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 03:05:10,85 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 03:05:10,85 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 03:05:10,85 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3552c2e1 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3552c2e1 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@526e7b0 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@8a0e7b9, splicing queue 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@526e7b0 queue spliced 1 messages 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#1327625029], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:10,86 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1181377650] 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:10,86 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 20000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 03:05:10,87 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 03:05:10,87 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=69}] 03:05:10,87 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 3 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 3 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data Simple{size=69} 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 03:05:10,88 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:05:10,88 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 03:05:10,88 PM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 40.01 μs 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 3, lastIndex: 4, isVoting: true 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 4, nextIndex: 5 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 4 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 3 to 4 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@6a728a0a 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,89 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} not found in inflight queue, checking pending queue 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 3 for term 4 03:05:10,89 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:05:10,89 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1505457766]}, enqueuedTicks=956490068962, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1}} 03:05:10,89 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE}: purge completed 03:05:10,89 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@70eca39f purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 4, lastIndex: 4, isVoting: true 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Follower's log matches - sending ElectionTimeout 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leader transfer complete 03:05:10,89 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=4 03:05:10,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 03:05:10,89 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader] 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=13] 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 4 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 4 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 3 to 4 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64} 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 3 for term 4 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:05:10,90 PM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=4 03:05:12,103 PM [cluster-test-shard-dispatcher-30] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Successfully transferred leadership to member-1-shard-cars-testTransactionWithIsolatedLeader in 2.017 s 03:05:12,103 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer succeeded - sending PoisonPill 03:05:12,103 PM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 03:05:12,103 PM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-449: Closing 03:05:12,103 PM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 03:05:12,104 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 03:05:12,104 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 03:05:12,104 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:05:12,104 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 03:05:12,104 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:05:12,105 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:05:12,105 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:05:12,105 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:05:12,108 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:12,108 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:12,108 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:12,109 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:12,111 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:05:12,124 PM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 03:05:12,127 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 03:05:12,127 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 03:05:12,127 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 03:05:12,128 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 03:05:12,128 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:05:12,128 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:05:13,124 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4063228306069740001] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:05:13,145 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2677380883864797108] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:05:13,148 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:13,148 PM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:13,150 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:05:13.152] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:05:13,159 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:05:13,159 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:05:13,160 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:05:13.161] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:05:13,170 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:05:13,174 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1607731639208026127] 03:05:13,175 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 03:05:13,176 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 03:05:13,176 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 03:05:13,178 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:05:13,178 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:05:13,186 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 03:05:13,186 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 03:05:13,186 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 03:05:13,188 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 03:05:13,193 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8287234317655758673] 03:05:13,193 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 03:05:13,194 PM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:05:13,194 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 03:05:13,206 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:05:13,206 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:05:13,213 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 03:05:13,223 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4393e233 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} 03:05:13,223 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 1 03:05:13,223 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-311338423], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@31b1ee31 03:05:13,223 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:05:13,223 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1320733959], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:05:13,223 PM [Cleaner-1] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@14b8949 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 03:05:13,223 PM [Cleaner-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-311338423]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2084/0x00000008016bea50@3ccf1288 03:05:13,223 PM [Cleaner-1] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1089983079], sessionId=2, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 03:05:13,233 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-968587975446880003] 03:05:13,233 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 03:05:13,237 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:05:13,237 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 03:05:13,245 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 03:05:13,245 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 03:05:13,245 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:05:13,246 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:05:13,248 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 03:05:13,254 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 03:05:13,256 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 03:05:13,256 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:05:13,256 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4d9d0d39 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@89f8aee 03:05:13,256 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6424f29a 03:05:13,256 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1709823913256, None)) 03:05:13,256 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823913255,0,0)) 03:05:13,256 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@89f8aee to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@68b075a1 03:05:13,256 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 03:05:13,257 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:05:13,258 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:05:13,258 PM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 03:05:13,260 PM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 03:05:13,261 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-164787142] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,262 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 03:05:13,264 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 03:05:13,265 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 03:05:13,268 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 03:05:13,268 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2a8b7dc6 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,269 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,270 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 03:05:13,270 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,270 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 03:05:13,270 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 03:05:13,270 PM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#2012960190 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,270 PM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1958086606] 03:05:13,270 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 03:05:13,270 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,271 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:05:13,271 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:05:13,271 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:05:13,271 PM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:05:13,271 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:05:13,271 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 03:05:13,271 PM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:05:13,271 PM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 03:05:13,271 PM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 03:05:13,271 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 03:05:13,271 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 03:05:13,271 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 03:05:13,271 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 03:05:13,271 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 03:05:13,272 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#2012960190] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,272 PM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1958086606] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,274 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#2050470192] 03:05:13,275 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#2050470192] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,275 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 03:05:13,281 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 03:05:13,281 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:05:13,281 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@130f8dec to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@649c410e 03:05:13,281 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@22d67e75 03:05:13,281 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1709823913281, None)) 03:05:13,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1579228028] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,289 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1709823913280,0,0)) 03:05:13,289 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@649c410e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5663f9cb 03:05:13,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-949868824 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 03:05:13,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 03:05:13,289 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-949868824] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,289 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 03:05:13,290 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 03:05:13,290 PM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@466559ef 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 03:05:13,292 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-2-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-326458716] 03:05:13,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:13,293 PM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:13,293 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-2-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#-968966168] 03:05:13,293 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 03:05:13,293 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-326458716] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,294 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1667220639 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,294 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 03:05:13,294 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 03:05:13,295 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 03:05:13,295 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1667220639] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,296 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#-968966168] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,296 PM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#2136264105] 03:05:13,296 PM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#2136264105] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,296 PM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#2107648174] 03:05:13,297 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#2107648174] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,298 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#587344991 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:13,298 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#587344991] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:13,298 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:05:13,298 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 03:05:13,299 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 03:05:13,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 03:05:13,299 PM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:05:13,299 PM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 03:05:13,299 PM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,300 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 03:05:13,300 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 03:05:13,300 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 03:05:13,303 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 03:05:13,303 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 03:05:13,341 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-164787142] 03:05:13,342 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 03:05:13,342 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442] 03:05:13,517 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:05:13,517 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:05:13,518 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 03:05:13,518 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,518 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 03:05:13,518 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:05:13,518 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 03:05:13,518 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:05:13,518 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 03:05:13,518 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 03:05:13,518 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 03:05:13,527 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 03:05:13,527 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 03:05:13,527 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 03:05:13,527 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,528 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 03:05:13,528 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 03:05:13,528 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 03:05:13,528 PM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 03:05:13,528 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 03:05:13,528 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 03:05:13,528 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 03:05:13,767 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:05:13,767 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:13,767 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 03:05:13,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:05:13,787 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:13,787 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 03:05:13,857 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:05:13,886 PM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 03:05:14,37 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:05:14,37 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,37 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 03:05:14,68 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:05:14,68 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:14,68 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 03:05:14,189 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 03:05:14,189 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,190 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,196 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,255 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 03:05:14,297 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:05:14,297 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:14,297 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 03:05:14,297 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:05:14,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 03:05:14,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:05:14,299 PM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:05:14,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 03:05:14,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:05:14,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:14,299 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:05:14,300 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:05:14,300 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:05:14,300 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 03:05:14,300 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:05:14,300 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:05:14,300 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:05:14,300 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 03:05:14,301 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 03:05:14,301 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:05:14,301 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:05:14,301 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:05:14,301 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 03:05:14,301 PM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 03:05:14,301 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:14,301 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:05:14,301 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:05:14,301 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:05:14,301 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 03:05:14,302 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:05:14,302 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:05:14,328 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 03:05:14,328 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,328 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 03:05:14,328 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 03:05:14,329 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 03:05:14,330 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 03:05:14,330 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:05:14,330 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:05:14,330 PM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:05:14,330 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 03:05:14,330 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 03:05:14,330 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 03:05:14,330 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,330 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=13] 03:05:14,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:05:14,330 PM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:05:14,330 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 03:05:14,330 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 03:05:14,331 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 03:05:14,331 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 03:05:14,398 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-164787142] 03:05:14,398 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 03:05:14,399 PM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370] 03:05:14,399 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@649ca9e0: creating a new transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:05:14,399 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 03:05:14,399 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:14,399 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:14,399 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1579228028] 03:05:14,399 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:14,399 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 03:05:14,399 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:05:14,399 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:05:14,399 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 03:05:14,400 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=13] 03:05:14,400 PM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 03:05:14,400 PM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:05:14,400 PM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:05:14,400 PM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1579228028] 03:05:14,400 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4b23bb55, attempting to connect 03:05:14,400 PM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 03:05:14,400 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:05:14,400 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:05:14,400 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:05:14,400 PM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@649ca9e0 readied transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 03:05:14,400 PM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 03:05:14,400 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@101d52d0 03:05:14,400 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 03:05:14,400 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2038/0x00000008016a13b0@6d86498d 03:05:14,400 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 03:05:14,401 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=13] 03:05:14,401 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4e4939b4, attempting to connect 03:05:14,406 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 03:05:14,406 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 03:05:14,406 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=15510, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:05:14,406 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=14480, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:05:14,407 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:05:14,407 PM [ForkJoinPool.commonPool-worker-3] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 03:05:14,407 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@149305fa instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@9948977 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@9948977 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@6a2a5d33 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@163ed977 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=960806353449} 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-453: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE}} 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-453: Message size: 395 bytes, total slices to send: 1 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-453: Message does not need to be sliced - sending original message 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@649ca9e0 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@60fcbebb 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@194491a1 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@25a5d85e, splicing queue 03:05:14,408 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@194491a1 queue spliced 0 messages 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 982.9 μs 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@611c8b3e instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ce8c725 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ce8c725 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2045/0x00000008016a79e0@578493ce 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3434aaaf transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=960806224025} 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-453: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE}} 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-453: Message size: 392 bytes, total slices to send: 1 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-453: Message does not need to be sliced - sending original message 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@649ca9e0 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6063df2e 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1ece5651 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@61751b0d, splicing queue 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1ece5651 queue spliced 0 messages 03:05:14,409 PM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 727.5 μs 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 03:05:14,410 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 03:05:14,410 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 03:05:14,411 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 03:05:14,411 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=960806224025, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 03:05:14,411 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 03:05:14,411 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 03:05:14,411 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=960806353449, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 03:05:14,412 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 03:05:14,412 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 03:05:14,412 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@745d744d 03:05:14,412 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 03:05:14,412 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 03:05:14,412 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@750f108f 03:05:14,412 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:05:14,412 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3434aaaf transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960817931699} 03:05:14,412 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 03:05:14,412 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2050/0x00000008016a9708@f7e472c 03:05:14,412 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 03:05:14,412 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 03:05:14,412 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2052/0x00000008016a9b78@506bbd6f 03:05:14,412 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:05:14,412 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@163ed977 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960818158134} 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:05:14,413 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 03:05:14,413 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 03:05:14,414 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 03:05:14,414 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960817931699, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 03:05:14,414 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:05:14,414 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 03:05:14,414 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960818158134, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 03:05:14,414 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 03:05:14,414 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 03:05:14,414 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@4f8b5433 03:05:14,414 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 03:05:14,414 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 03:05:14,414 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@4f8b5433 03:05:14,414 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:05:14,414 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3434aaaf transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960820048262} 03:05:14,414 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 03:05:14,414 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@420dbe25 03:05:14,414 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 03:05:14,414 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 03:05:14,414 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2053/0x00000008016aaea8@420dbe25 03:05:14,414 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:05:14,414 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@163ed977 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960820168875} 03:05:14,414 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 03:05:14,414 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,415 PM [cluster-test-shard-dispatcher-31] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 03:05:14,415 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 03:05:14,415 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:05:14,415 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 03:05:14,415 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:05:14,415 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,415 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 03:05:14,415 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 03:05:14,415 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 03:05:14,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 03:05:14,416 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:05:14,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 03:05:14,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:05:14,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:05:14,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 03:05:14,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:05:14,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 03:05:14,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 03:05:14,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:05:14,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 03:05:14,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=176} 03:05:14,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 03:05:14,416 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:05:14,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 03:05:14,416 PM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:05:14,417 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 03:05:14,417 PM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@34a825f9}, purged=MutableUnsignedLongSet{size=0}} 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#920961658] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:05:14,417 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#920961658] 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@527d5257}, purged=MutableUnsignedLongSet{size=0}} 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1111902299] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#920961658] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 65.31 μs 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1111902299] 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 03:05:14,418 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960820168875, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@149305fa completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#1111902299] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@249e6904 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@5a7a4816 03:05:14,418 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:05:14,419 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@163ed977 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960824487043} 03:05:14,419 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,419 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 03:05:14,419 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 03:05:14,419 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:05:14,420 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 128.8 μs 03:05:14,421 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 03:05:14,421 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960820048262, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@611c8b3e completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 03:05:14,422 PM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@649ca9e0: creating a new snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@36eaf1d6 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$2055/0x00000008016b4000@7f0e2975 03:05:14,422 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:05:14,422 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:05:14,422 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@3f37d66f 03:05:14,422 PM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3434aaaf transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960827710594} 03:05:14,422 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3434aaaf transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=960827899128} 03:05:14,422 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 03:05:14,422 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:05:14,422 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 03:05:14,422 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:05:14,422 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 03:05:14,422 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,422 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 03:05:14,422 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 03:05:14,422 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 03:05:14,422 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 03:05:14,422 PM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 03:05:14,423 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 03:05:14,423 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 03:05:14,423 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4d37f16f}, purged=MutableUnsignedLongSet{size=0}} 03:05:14,423 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@96b972c}, purged=MutableUnsignedLongSet{size=0}} 03:05:14,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 03:05:14,423 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 03:05:14,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,423 PM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,423 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-688623058] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 03:05:14,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-2025568095] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 03:05:14,423 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:05:14,423 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-688623058] 03:05:14,423 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:05:14,423 PM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-688623058] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:14,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-2025568095] 03:05:14,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 03:05:14,423 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 109.6 μs 03:05:14,423 PM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 40.67 μs 03:05:14,423 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-2025568095] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:14,424 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 03:05:14,429 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,429 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 03:05:14,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 03:05:14,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 03:05:14,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 03:05:14,429 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 03:05:14,429 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 03:05:14,429 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 03:05:14,429 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-452: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 03:05:14,430 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-452: Message size: 333 bytes, total slices to send: 1 03:05:14,430 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-452: Message does not need to be sliced - sending original message 03:05:14,430 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 03:05:14,430 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:05:14,430 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:05:14,430 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 03:05:14,430 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 03:05:14,430 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 03:05:14,430 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:05:14,430 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 03:05:14,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:05:14,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:05:14,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 03:05:14,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:05:14,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 03:05:14,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@3a008ddd 03:05:14,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,430 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 03:05:14,431 PM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,431 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:05:14,431 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:05:14,431 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:05:14,431 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 03:05:14,431 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:05:14,431 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 03:05:14,431 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:05:14,431 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 03:05:14,431 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 03:05:14,432 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 03:05:14,432 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:05:14,432 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=960827899128, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960824487043, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@9948977 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@149305fa purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 03:05:14,432 PM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 03:05:14,432 PM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 03:05:14,432 PM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2057/0x00000008016b4810@32ba9423 03:05:14,432 PM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:05:14,432 PM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@163ed977 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=960838133995} 03:05:14,432 PM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 03:05:14,432 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-451: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 03:05:14,433 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-451: Message size: 336 bytes, total slices to send: 1 03:05:14,433 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-451: Message does not need to be sliced - sending original message 03:05:14,433 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 03:05:14,433 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1400312539], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=960838133995, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 03:05:14,433 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 03:05:14,433 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:05:14,433 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,433 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:14,433 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@d2dde8c 03:05:14,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 03:05:14,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@d2dde8c 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 03:05:14,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ab9859a 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5ab9859a 03:05:14,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 03:05:14,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:05:14,434 PM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 03:05:14,434 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=13, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 03:05:14,434 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 03:05:14,434 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 03:05:14,434 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 03:05:14,434 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 03:05:14,434 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards] 03:05:14,434 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=13, raftVersion=5, recipientRaftVersion=5] 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@163ed977 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4284e2fd, splicing queue 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@163ed977 queue spliced 0 messages 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: refreshing backend for shard 2 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-1765178370], sessionId=0, version=POTASSIUM, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1579228028] 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards] 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=13] 03:05:14,435 PM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3434aaaf superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@10fd9948, splicing queue 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3434aaaf queue spliced 1 messages 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: refreshing backend for shard 1 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442], sessionId=1, version=POTASSIUM, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 03:05:14,435 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1579228028] 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-people-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2058/0x00000008016b4ed0@28f45210 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testWriteTransactionWithMultipleShards, matchIndex: 1, lastIndex: 1, isVoting: true 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Follower's log matches - sending ElectionTimeout 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leader transfer complete 03:05:14,436 PM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:05:14,436 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Request for Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} not found in inflight queue, checking pending queue 03:05:14,436 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 03:05:14,436 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1400312539]}, enqueuedTicks=960827710594, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3}} 03:05:14,436 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE}: purge completed 03:05:14,436 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ce8c725 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} 03:05:14,436 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 03:05:14,436 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@611c8b3e purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68} 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 03:05:14,436 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 03:05:14,437 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 03:05:14,847 PM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 03:05:14,847 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1104931442]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 03:05:14,847 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,847 PM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 03:05:14,847 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-457: Closing 03:05:14,848 PM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-456: Closing 03:05:14,848 PM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:05:14,848 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:05:14,848 PM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 03:05:14,848 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 03:05:14,848 PM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 03:05:14,848 PM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 03:05:14,849 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: shutting down 03:05:14,849 PM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 03:05:14,849 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-164787142] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-164787142] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 03:05:14,849 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:05:14,849 PM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:05:14,854 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:14,854 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:14,854 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:14,855 PM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:14,857 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 03:05:14,869 PM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 03:05:14,869 PM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 03:05:14,869 PM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 03:05:14,869 PM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 03:05:14,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 03:05:14,869 PM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:05:14,869 PM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 03:05:14,869 PM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 03:05:14,871 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:05:14,871 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:05:15,864 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1607731639208026127] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:05:15,884 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8287234317655758673] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 03:05:15,889 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:15,889 PM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 03:05:15,893 PM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:05:15.894] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 03:05:15,899 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 03:05:15,899 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 03:05:15,900 PM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [03/07/2024 15:05:15.901] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config